Integrating Processing

This guide explains how to add a custom Audio Processing component with MimiCore inside MimiSDK, providing your users with Mimified audio.

Activation

As the MimiProcessingController owns the MimiProcessingSession; it is the location where processing can be “activated” or “deactivated”.

The lifecycle of the MimiProcessingSession instance should align with the headphone connectivity lifecycle. The activation of the session should happen once a connection with the headphones has been established and the techLevel is obtained from the firmware, since it is required for the session activation. Similarly, the session should be deactivated once the headphones have been disconnected.

Note: Activation refers to the ability for parameters values to be transferred and not whether the physical processing effect is enabled or disabled.

MimiBasicProcessingConfiguration

Activating a MimiProcessingSession requires a MimiBasicProcessingConfiguration. This configuration allows you to specify the personalization mode and define applicators for processing parameters including isEnabled, intensity, and preset.

Personalization Configuration

Currently, two Personalization modes i.e. FineTuning (which is the recommended option) and SinglePreset are supported. The FineTuning mode provides up to three presets (up, down and default) whereas the SinglePreset mode provides one personalization preset. The presets are based on the user’s hearing profile.

Note: The three (up, down and default) FineTuning presets are available depending on the user’s hearing profile and there may be cases where it is not possible to generate the either the up preset or the down preset. In such cases. However, the default preset will be available.

Session Activation with a MimiBasicProcessingConfiguration

Before activating a MimiProcessingSession, the preset Tech-Level should be read from the Mimi Processing System on the device/headphones.

A MimiProcessingSession for a Fine Tuning type of Personalization can be activated by defining a MimiBasicProcessingConfiguration as follows:

do {
    // Read Tech-Level from the Mimi Processing System
    let fitting = MimiPersonalization.Fitting.techLevel(4) // An example fitting, yours will be defined by the Mimi Processing system on the device.

    let configuration = try MimiBasicProcessingConfiguration {
        SoundPersonalization {
            FineTuning(fitting: fitting)

            Applicators {
                IsEnabled { value in
                    // Apply the isEnabled value to your audio processing system
                }

                Intensity { value in
                    // Apply the intensity value to your audio processing system
                }

                Preset { value in
                    // Apply the preset value to your audio processing system
                }
            }
        }
    }
    let session = try await MimiCore.shared.processing.activate(configuration: configuration)
} catch {
    // Handle activation failure
}

Application timeout

It is also possible to set a timeout for the application of a value in the IsEnabled, Intensity and Preset applicators. If the value application exceeds this timeout, it will cause the entire value application sequence to fail.

let configuration = try MimiBasicProcessingConfiguration {
    SoundPersonalization {
        FineTuning(fitting: fitting)

        Applicators {
            IsEnabled(timeout: 0.5) { value in
                // Apply the isEnabled value or throw an error.
                // If the timeout is exceeded, it causes the value application sequence to fail
                // and throws the ParameterError.applyTimeoutExceeded error.
            }

            Intensity { value in
                // Apply the intensity value to your audio processing system
            }

            Preset { value in
                // Apply the preset value to your audio processing system
            }
        }
    }
}

Once there is an activated MimiProcessingSession, it is possible to access the processing parameters and configure them appropriately.

  • Activating a session will set the session property on the MimiProcessingController allowing the session to be easily referenced.
  • Only a single session can be activated at one time.
  • MimiProcessingController.sessionPublisher is a AnyPublisher and can be subscribed for changes that occur during the activation & deactivation of a session.

Tip: Most partner integrations will only need to use a single MimiProcessingSession that roughly aligns with the headphone connectivity lifecycle.

Parameter Value Operations

Observing the current value

The valuePublisher property of the MimiProcessingParameter can be subscribed to as follows.

Example usage:

let subscription = session.soundPersonalization?.media?.isEnabled.valuePublisher
    .sink { value in
        print("isEnabled parameter value: \(value)")
}

Observing State Changes

This property enables observation of state changes in the ProcessingParameter resulting from a value application process. It emits values of type MimiProcessingParameterUpdateState to notify subscribers about any changes.

Example usage:

   // Subscribe to states changes of the isEnabled processing parameter
let subscription = session.soundPersonalization?.media?.isEnabled.updateState
    .sink { state in
        print("isEnabled parameter update state: \(state)")
}

Updating the value

The parameter value can be updated via the apply(value:) method. This method will attempt to apply the provided value on the associated applicator. For the application to succeed, success must be reported by the applicator that the application is attempted on. If the applicator fails to report success, the application fails, and an error is thrown.

Example usage:

do {
    try await session.soundPersonalization?.media?.isEnabled.apply(value)
    // Value application succeeded
} catch {
    // Handle error
}

Note: For remote parameters e.g. preset, use the load() method which will attempt to load a new parameter value from a remote data source and then apply it on all registered applicators.


## Advanced
### Synchronizing Applicator

The `synchronizeApplicator()` function attempts to update the associated Applicator with the current value of the `MimiProcessingParameter`, if the Applicator is out-of-date.

### Parameter Recommendations
Based on certain events in `MimiSDK` e.g. user authentication, hearing test submission etc, the processing parameters are automatically set to recommended values.