Processing¶

This guide explains how to add a custom Audio Processing component with MimiCore inside MSDK, providing your users with "Mimified" audio.

Architecture Overview¶

The Processing APIs can be divided into four main components:

Processing Controller - The root entry point and context for Processing API access.
Processing Session - The provider of access to processing Processing Parameters when activated.
Processing Parameters - Individual parameter objects that provides state, data and mutation.
Processing Applicators - Responsible for applying the Processing Parameter values to external systems.

The following section describes the components in the Processing APIs.

Processing System Overview

Figure: Processing System Components

ProcessingController¶

Overview¶

The ProcessingController is a controller owned and initialized by MimiCore.

It is responsible for the activation and deactivation of Processing Sessions, and therefore can be used as the source of truth for the active ProcessingSession instance.

Responsibilities¶

Centralized location for accessing Processing APIs.
Owner of active Processing Session
Provides activation and deactivation of Processing Session.

Activation¶

As the ProcessingController owns the ProcessingSession status; it is the location where processing can be "activated" or "deactivated".

Once there is an activated ProcessingSession, it is possible to access the MimiProcessingParameters and configure them appropriately for use.

Lifecycle¶

The ProcessingController is responsible for handling the ProcessingSession lifecycle in terms of activation and deactivation.

Activating a session will set the activeSession property on the ProcessingController allowing the session to be easily referenced.
Only a single session can be activated at one time.
Deactivation clears the activeSession property. It does not directly affect the applied Mimi processing.
Activating and deactivating a session triggers MimiObserver notifications to observers.

ProcessingSession¶

Provides access to the Processing Parameters.

Overview¶

As described in the previous section, a ProcessingSession is owned by a ProcessingController.

The ProcessingSession itself owns the MimiProcessingParameters, and therefore is the gateway to parameter access.

Responsibilities¶

Owner of all Processing Parameters, providing access to; isEnabled, intensity and preset.
Exposes the current data source configuration for the preset parameter.
Provides the ability for processing to be "interrupted" and automatically disabled due to external events (such as the hearing test launching).

Upon activation, the ProcessingSession will be initialized and the previously applied parameter values are restored to their respective parameters. At this point the parameters are ready for use.

Processing Parameter¶

The focal point for state.

Overview¶

A MimiProcessingParameter represents an individual parameter that can be used to control Mimi processing functionality. It encompasses a value that can be mutated and applied asynchronously to subscribed components called Applicators.

Responsibilities¶

In general terms, a MimiProcessingParameter is responsible for:

Providing access to the current parameter value.
Applying values to an Applicator using the value application sequence.
Loading values from remote data sources asynchronously (such as preset data).
Providing state and value updates to a collection of subscribed observers.

MSDK ProcessingParameter instances¶

The ProcessingSession defines the following MimiProcessingParameter instances to provide access and control of the Mimi processing:

isEnabled - Whether Mimi processing is enabled or disabled.
intensity - Intensity of Mimi processing.
preset - Preset data model for use by a Mimi processor.

Parameter value application sequence summary¶

The internal process of updating a MimiProcessingParameter value or transferring the existing value to the Applicator follows a specific sequence as executed by the MimiProcessingParameter:

The Applicator is requested to apply the new value.
- If any Applicator fails to apply the value (by returning MimiApplicatorResult.Failure), the application sequence fails and observers are notified of the failure.
- If successful (the Applicator returns MimiApplicatorResult.Success), the value is updated and observers are notified of the new value.
A return result value is given for the request.

Applicator¶

Arguably the most relevant Processing API component for integrators is the Applicator.

An Applicator is an accessory of a MimiProcessingParameter that is capable of performing the value application logic for its associated MimiProcessingParameter.

In simple terms, its role is to apply MimiProcessingParameter values to an external system. When there is a request to update a MimiProcessingParameter value, its Applicator is requested to apply this value.

For example, it will send Bluetooth commands to your device to set the isEnabled state.

While an Applicator appears quite similar to an Observer, the two components have distinct roles:

An Applicator is responsible for the value application logic of a MimiProcessingParameter - the result of which influences the MimiProcessingParameter state.
An Observer is only informed of events reflecting committed changes to the MimiProcessingParameter state; it cannot influence the MimiProcessingParameter state.

Responsibilities¶

Provides an onApply function which will allow the Applicator to asynchronously apply a value your processing system, returning the appropriate MimiApplicatorResult instance.
Ability to set a “Delivery Timeout” to specify a time duration which if exceeded is considered erroneous and will cause a value application sequence to fail.

Getting Started¶

This section highlights some important APIs used to integrate your processing system with the MSDK.

Configuration and Activation¶

When activating a ProcessingSession, you need to provide a MimiProcessingConfiguration to define its configuration.

Example

Activate a ProcessingSession with a MimiProcessingConfiguration

First define a MimiProcessingConfiguration instance:

val configuration = MimiProcessingConfiguration.Basic(TODO("Define your configuration here"))

Then activate the ProcessingSession:

val session : Result<ProcessingSession> = MimiCore.processingController.activateSession(configuration)

`MimiProcessingConfiguration` Overview¶

The MimiProcessingConfiguration defines the configuration of a ProcessingSession and provides the ability to configure the personalization preset data source.

It can be expressed directly through class instantiation, or using the fluent DSL syntax.

Example

Define MimiProcessingConfiguration with "Fine Tuning" personalization:

val configuration = MimiProcessingConfiguration.Basic(
    soundPersonalization = SoundPersonalizationConfiguration(
        mode = PersonalizationModeConfiguration.FineTuning(Fitting(TODO("Fitting is defined by your Processing system"))),
        parameterConfiguration = SoundPersonalizationParametersConfiguration(
            isEnabled = ProcessingParameterConfiguration(
                YOUR_APPLY_TIMEOUT, // TODO Define a timeout appropriate for your system
                isEnabledApplicator::apply // TODO Reference your isEnabled Applicator function
            ),
            intensity = ProcessingParameterConfiguration(
                YOUR_APPLY_TIMEOUT,
                intensityApplicator::apply // TODO Reference your intensity Applicator function
            ),
            preset = ProcessingParameterConfiguration(
                YOUR_APPLY_TIMEOUT,
                presetApplicator::apply // TODO Reference your preset Applicator function
            ),
        )
    )
)

Warning

The Processing Configuration DSL is experimental and requires opt-in, as it may have unannounced changes in future releases.

Read more in the Preset Parameter Data Sources guide.

Synchronizing your Processor¶

When a ProcessingSession is activated, an Applicator will not automatically receive the current MimiProcessingParameter value. For the MimiProcessingParameter to receive the initial value, you need to call synchronizeApplicator() for each Parameter.

Example

After activating the ProcessingSession

// Make each Applicator receive the current value from their respective ProcessingParameter
val isEnabledSyncResult : ProcessingParameterResult? = session.soundPersonalization?.media?.isEnabled?.synchronizeApplicator()
val presetSyncResult : ProcessingParameterResult? = session.soundPersonalization?.media?.preset?.synchronizeApplicator()
val intensitySyncResult : ProcessingParameterResult? = session.soundPersonalization?.media?.intensity?.synchronizeApplicatorr()

Common `MimiProcessingParameter` operations¶

`MimiProcessingParameter` Value operations¶

The following operations are provided on the MimiProcessingParameter and execute the value application sequence, and return a ProcessingParameterResult indicating whether the operation was successful.

Function	Behavior	Notes
`apply(value)`	The `apply` function attempts to update the `MimiProcessingParameter` with the given `value` by updating all registered, out-of-date `Applicator` instances.
`synchronizeApplicator()`	The `synchronizeApplicator` function attempts to update the `Applicator` instance with the current `MimiProcessingParameter` `value`.
`load()`	The `load` function will attempt to refresh the value from the data source, and then automatically execute the value application sequence with that value.	Only available for applicable if the `MimiProcessingParameter` has a `MimiProcessingParameterDataSource`

Reading the `MimiProcessingParameter` state¶

Property/Function	Behavior	Notes
`value`	The `value` property holds the latest successfully applied Parameter value. It is updated at the end of the value application sequence
`observe()`	An `Observer` which provides callbacks for observing and value and state changes related to the `MimiProcessingParameter`. It has the following events: `Applying` - the request value is currently being applied to the Applicators. `Failed` - the most recent value application failed. `Applied` - the most recent value application was successful. `Loading` - the value is being retrieved from the data source.	*Important: You should not* use `observe` to apply values to your processing system; that is the responsibility of an `Applicator`.**

Example

Get the current parameter value:

val activeSession : ProcessingSession = requireNotNull(MimiCore.processingController.activeSession.state.value)
val isEnabled : Boolean = activeSession.soundPersonalization?.media?.isEnabled?.value

Example

Observe changes in the MimiProcessingParameter state:

// Acquire the active ProcessSession (assumes already activated!)
val activeSession : ProcessingSession = requireNotNull(MimiCore.processingController.activeSession.state.value)

// Observer for a MimiProcessingParameter
activeSession.soundPersonalization?.media?.isEnabled?.observe {
   when(val updateState = it.updateState) {
      ProcessingParameterUpdateState.Applied -> TODO() // add your code here to handle when value application succeeds
      is ProcessingParameterUpdateState.Applying -> TODO() // add your code here to handle when a value application is in progress
      is ProcessingParameterUpdateState.Failed -> TODO() // add your code here to handle when a value application fails
      ProcessingParameterUpdateState.Loading -> TODO() // add your code here to handle when a value application is loading from the data source
   }
}

Processing¶

Architecture Overview¶

ProcessingController¶

Overview¶

Responsibilities¶

Activation¶

Lifecycle¶

ProcessingSession¶

Overview¶

Responsibilities¶

Processing Parameter¶

Overview¶

Responsibilities¶

MSDK ProcessingParameter instances¶

Parameter value application sequence summary¶

Applicator¶

Responsibilities¶

Getting Started¶

Configuration and Activation¶

MimiProcessingConfiguration Overview¶

Synchronizing your Processor¶

Common MimiProcessingParameter operations¶

MimiProcessingParameter Value operations¶

Reading the MimiProcessingParameter state¶

`MimiProcessingConfiguration` Overview¶

Common `MimiProcessingParameter` operations¶

`MimiProcessingParameter` Value operations¶

Reading the `MimiProcessingParameter` state¶