Integrating Processing

This guide explains how to add a custom Audio Processing component with MimiCore inside MimiSDK, providing your users with Mimified audio.

Activation

As the MimiProcessingController owns the MimiProcessingSession; it is the location where processing can be “activated” or “deactivated”.

The lifecycle of the MimiProcessingSession instance should align with the headphone connectivity lifecycle. The activation of the session should happen once a connection with the headphones has been established and the techLevel is obtained from the firmware, since it is required for the session activation. Similarly, the session should be deactivated once the headphones have been disconnected.

Note: Activation refers to the ability for parameters values to be transferred and not whether the physical processing effect is enabled or disabled.

MimiProcessingConfiguration

Activating a MimiProcessingSession requires a MimiProcessingConfiguration. Currently, only the type of Personalization can be configured in the Configuration API. Support for additional configurations e.g. Processing Parameter configurations is planned for in the future.

Personalization Configuration

Currently, two Personalization modes i.e. FineTuning (which is the recommended option) and SinglePreset are supported. The FineTuning mode provides up to three presets (up, down and default) whereas the SinglePreset mode provides one personalization preset. The presets are based on the user’s hearing profile.

Note: The three (up, down and default) FineTuning presets are available depending on the user’s hearing profile and there may be cases where it is not possible to generate the either the up preset or the down preset. In such cases. However, the default preset will be available.

Session Activation with a MimiProcessingConfiguration

Before activating a MimiProcessingSession, the preset Tech-Level should be read from the Mimi Processing System on the device/headphones.

A MimiProcessingSession for a Fine Tuning type of Personalization can be activated by defining a MimiProcessingConfiguration as follows:

do {
    // Read Tech-Level from the Mimi Processing System
    let fitting = MimiPersonalization.Fitting.techLevel(4) // An example fitting, yours will be defined by the Mimi Processing system on the device.

    let configuration = mimiProcessingConfiguration {
                            Personalization {
                                FineTuning(fitting: fitting)
                            }
                        }
    let session = try await MimiCore.shared.processing.activate(configuration: configuration)
} catch {
    // Handle activation failure
}

A MimiProcessingSession for a Single Preset type of Personalization can be activated by defining a MimiProcessingConfiguration as follows:

do {
    // Read Tech-Level from the Mimi Processing System
    let fitting = MimiPersonalization.Fitting.techLevel(4) // An example fitting, yours will be defined by the Mimi Processing system on the device.

    let configuration = mimiProcessingConfiguration {
                            Personalization {
                                SinglePreset(fitting: fitting)
                            }
                        }
    let session = try await MimiCore.shared.processing.activate(configuration: configuration)
} catch {
    // Handle activation failure
}

Once there is an activated MimiProcessingSession, it is possible to access the processing parameters and configure them appropriately.

  • Activating a session will set the session property on the MimiProcessingController allowing the session to be easily referenced.
  • Only a single session can be activated at one time.
  • MimiProcessingController.sessionPublisher is a AnyPublisher and can be subscribed for changes that occur during the activation & deactivation of a session.

Tip: Most partner integrations will only need to use a single MimiProcessingSession that roughly aligns with the headphone connectivity lifecycle.

Parameter Value Operations

Observing the current value

The valuePublisher property of the MimiProcessingParameter is published var and can be subscribed to as follows.

cancellable = parameter.valuePublisher
    .sink { value in
        // Do something with value
    }

Observing State Changes

This property enables observation of state changes in the ProcessingParameter resulting from a value application process. It emits values of type MimiProcessingParameter.ParameterUpdateState to notify subscribers about any changes.

Example usage:

   // Subscribe to states changes of the isEnabled processing parameter
   let subscription = session.isEnabled.updateState
        .sink { state in
           print("isEnabled parameter update state: \(state)")
   }

Value application sequence summary

The internal process of updating a MimiProcessingParameter value or sending value updates to the associated applicators follows a specific sequence as executed by the MimiProcessingParameter:

  1. Value update is triggered with the apply(value:) method.
  2. The out-of-date applicators are filtered out from the list of associated applicators. If no out-of-date applicators are found, then the parameter is immediately updated with the value and the subscribers are notified of the success.
  3. If any out-of-date applicators are found, they are requested to apply the new value via their apply method.
  4. If any applicator fails to apply the value (by throwing an error), the application sequence is abandoned with an error and the parameter updateState is updated to failed(value: Value, error: Error).
  5. If all applicators apply the value successfully, the parameter value is updated and the parameter updateState is updated to applied.

Updating the value

The parameter value can be updated via the apply(value:) method. This method will attempt to apply the provided value on all registered applicators that are out-of-date. For the application to succeed, success must be reported by all applicators that the application is attempted on. If any of the applicators fail to report success, the application fails, and an error is thrown.

do {
    try await parameter.apply(value)
    // Value application succeeded
} catch {
    // Handle error
}

Note: For remote parameters, please use the load() method which will attempt to load a new parameter value from a remote data source and then apply it on all registered applicators.

Defining an Applicator

An applicator can be defined and added using the MimiProcessingParameter.addApplicator(applyTimeout:apply:) method. Since the applicator is held weakly by the parameter, you will need to maintain a strong reference to it. Note that the current parameter value is not supplied when adding an applicator and must be triggered by calling the synchronizeApplicators() method.

// Make sure to hold on to the applicator strongly.
let applicator = await parameter.addApplicator { value in
    // Apply the value or throw an error.
}

Application timeout

It is also possible to set a timeout for the application of a value on the MimiProcessingApplicator. If the value application exceeds this timeout, it will cause the entire value application sequence to fail.

// Make sure to hold on to the applicator strongly.
let applicator = await parameter.addApplicator(applyTimeout: 0.5,
                                                apply: { value in
    // Apply the value or throw an error. 
    // If the timeout is exceeded, it causes the value application sequence to fail and 
    // throws the ParameterError.applyTimeoutExceeded error.
})

Synchronizing Applicators

The synchronizeApplicators() function attempts to update all registered MimiProcessingParameterApplicator instances that are currently out-of-date with the current value of the MimiProcessingParameter.

Removing an Applicator

Since the applicator is held weakly by its associated parameter, simply setting the applicator to nil is enough to remove it from the parameter’s list of applicators.

Session Interruptions

The processing session provides the ability to be interrupted via the MimiProcessingSession.interrupt(reason:) method. Interrupting a session leads to its isEnabled parameter to be set to false. This value cannot be changed until all the session interruptions are resolved via the MimiProcessingSession.resolve(interruption:) method. Once all the interruptions have been resolved, the value of the isEnabled parameter is set back to its pre-interruption value.

// Interrupting a session
let interruption = "MySessionInterruption"

do {
    let isInterrupted = try await session.interrupt(reason: interruption)
    // Do something with isInterrupted
} catch {
    // handle error
}

// Resolving session interruption
do {
    let isInterrupted = try await session.resolve(interruption: interruption)
    // Do something with isInterrupted
} catch {
    // handle error
}

Note: Certain events in MimiSDK such as launching of the test flow also lead to the session being temporarily interrupted.

Parameter Recommendations

Based on certain events in MimiSDK e.g. user authentication, hearing test submission etc, the processing parameters are automatically set to recommended values.