Helper functions with auxiliary logic

Functions for loading streams (statically or dynamically).

get_processor_and_schema(source)

Get the schema and processor from the given class.

Parameters:
  • source (type) –

    The class to get the schema and processor from.
Returns:
  • tuple[Schema, Processor] –

    tuple[kemux.data.schema.base.Schema, kemux.data.processor.base.Processor]: The schema and processor from the given class.
Raises:
  • ValueError –

    If the given class does not contain a schema or processor.

load_input(input_class)

Load an input class from the given class.

Parameters:
  • input_class (type) –

    The class of the input, containing the Schema and Processor.
Returns:
  • InputProcessor –

    kemux.data.processor.input.InputProcessor: The input processor loaded from the given class.

load_output(output_class)

Load an output class from the given class.

Parameters:
  • output_class (type) –

    The class of the output, containing the Schema and Processor.
Returns:
  • OutputProcessor –

    kemux.data.processor.output.OutputProcessor: The output processor loaded from the given class.

load_outputs(outputs)

Load all the output classes from the given class.

Parameters:
  • outputs (type) –

    The class of the outputs, each containing a Schema and a Processor.
Returns:
  • dict[str, OutputProcessor] –

    dict[str, kemux.data.processor.output.OutputProcessor]: The output processors loaded from the given class.

load_stream_module(streams_dir, module_filename)

Load a stream module from the given directory.

Parameters:
  • streams_dir (str) –

    The path to the directory containing the Stream classes.

  • module_filename (str) –

    The name of the module file to be loaded.
Returns:
  • Stream –

    kemux.data.stream.Stream: The stream loaded from the given module file.
Raises:
  • ValueError –

    If the given module file is not a valid stream module.

load_streams(streams_dir)

Load all the streams from the given directory.

Parameters:
  • streams_dir (str) –

    The path to the directory containing the Stream classes.
Returns:
  • dict[str, Stream] –

    dict[str, kemux.data.stream.Stream]: The streams loaded from the given directory.
Raises:
  • ValueError –

    If the given directory is not a valid directory.