Jupyter integration module

This module allows to use MLDev together with Jupyter notebooks.

See NotebookRunner class that runs code cells from Jupyter notebook.

class JupyterStage(name='', notebook_pipeline='', outputs=None, compare_results=None)[исходный код]

Базовые классы: object

A main class for IPython notebooks pipeline stages

Accepts the following parameters:

Параметры
  • name – stage name

  • notebook_pipeline – a string in format <path_to_notebook>.<pipeline_name>

  • outputs – lists files and folders that this stage produces or files from which variables are taken for comparison

  • compare_results – if True, will load saved results and compare with received during the run, if False or None, results received during the run will be saved

prepare(stage_name)[исходный код]
run(stage_name)[исходный код]
class NotebookPipeline(stage_name, pipeline_config, outputs, compare)[исходный код]

Базовые классы: object

get_context_from_notebook()[исходный код]

Returns the dict with pipeline information from notebook

run()[исходный код]

Starts running pipeline

class NotebookRunner(stage_name, notebook, outputs, compare, nb_context)[исходный код]

Базовые классы: object

A class that runs Jupyter notebook cells with IPython InteractiveShell. Reads all cells that marked with „#%mldev“ comment, run them in the order given in nb_context. After run can save received variables or compare them to previously saved ones.

Accepts the following parameters:

Параметры
  • stage_name – stage name

  • notebook – Jupyter notebook parsed to dict

  • outputs – lists files and folders that this stage produces or files from which variables are taken for comparison

  • compare – if True, will load saved results and compare with received during the run, if False or None, results received during the run will be saved

  • nb_context – dict with cells pipeline, cell dependencies and variables for save or compare

compare_results()[исходный код]

Gets saved variables, compares them with variables received during the run

get_cell_code(cell_name)[исходный код]

Returns code of cell with name cell_name

get_mldev_code_cells()[исходный код]

Returns dict with all cells, marked with „#%mldev“ comment

get_results()[исходный код]

Returns dict with variables from „results“ field, which received during the run

run_cell_with_deps(cell_name)[исходный код]

Runs cells, checks if cell has dependencies and runs them first, throws exceptions

run_pipeline(seq_name)[исходный код]
save_results()[исходный код]

Saves dict with variables to file, if var is Figure - saves it to PNG file