Integrating Processing

This guide explains how to add a custom Audio Processing component with MimiCore inside MimiSDK, providing your users with Mimified audio.

Activation

As the MimiProcessingController owns the MimiProcessingSession; it is the location where processing can be “activated” or “deactivated”.

Note: Activation refers to the ability for parameters values to be transferred and not whether the physical processing effect is enabled or disabled.

Before activating a MimiProcessingSession, you will need to create a preset parameter data source. The SDK currently provides two preset parameter data sources which are explained in Preset Parameter Data Sources.

A MimiProcessingSession can be activated as follows:

do {
    let fitting = MimiPersonalization.Fitting.techLevel(3) // An example fitting, yours will be defined by your Processing system
    let presetDataSource = MimiDefaultPresetParameterDataSource(fitting: fitting)
    _ = try MimiCore.shared.processing.activate(presetDataSource: .default(presetDataSource))
} catch {
    // Handle activation failure
}

Once there is an activated MimiProcessingSession, it is possible to access the processing parameters and configure them appropriately.

  • Activating a session will set the session.value property on the MimiProcessingController allowing the session to be easily referenced.
  • Only a single session can be activated at one time.
  • MimiProcessingController.session is a CurrentValueSubject and can be subscribed for changes that occur during the activation & deactivation of a session.

Tip: Most partner integrations will only need to use a single, long-lived MimiProcessingSession that roughly aligns with the application lifecycle.

Parameter Value Operations

Observing the current value

The value property of the MimiProcessingParameter is published var and can be subscribed to as follows.

cancellable = parameter.$value
    .sink { value in
        // Do something with value
    }

Value application sequence summary

The internal process of updating a MimiProcessingParameter value or sending value updates to the associated applicators follows a specific sequence as executed by the MimiProcessingParameter:

  1. Value update is triggered with the apply(value:) method.
  2. Based on the delivery mode, the update is handled either immediately (continous) or debounced (discrete).
  3. The out-of-date applicators are filtered out from the list of associated applicators. If no out-of-date applicators are found, then the parameter is immediately updated with the value and the subscribers are notified of the success.
  4. If any out-of-date applicators are found, they are then asked to verify that they are capable of applying the value via their canApply method.
  5. If any out-of-date applicator fails the canApply verification (i.e. by returning false), the value application sequence is abandoned with an error and subscribers are notified of the failure.
  6. After all out-of-date applicators pass the canApply verification (i.e. by returning true), the applicators are requested to apply the new value via their apply method.
  7. If any applicator fails to apply the value (by returning a failure(Error)), the application sequence is abandoned with an error and the subscribers are notified of the failure.
  8. If the applicator applies the value successfully (by returning a success), the parameter value is updated and the subscribers are notified of the success.

Updating the value

The parameter value can be updated via the apply(value:) method. This method will attempt to apply the provided value on all registered applicators that are out-of-date. For the application to succeed, success must be reported by all applicators that the application is attempted on. If any of the applicators fail to report this, the overall application is considered a failure and will escape.

cancellable = parameter.apply(value)
    .sink { completion in
        // Do something with completion
    } receiveValue: { value in
        // Do something with value
    }

Note: For remote parameters, please use the reload() method which will attempt to reload a new parameter value from a remote data source and then apply it on all registered applicators.

Delivery Mode

The delivery mode of a MimiProcessingParameter describes how it handles requests to update the parameter value. By default, the updates are handled in a continuous manner, meaning they are delivered to the applicators immediately when the parameter is updated via the apply(value:) method. This can be customized to an alternate delivery mode if preferred:

  • .continuous (Default) - Executes parameter updates immediately.
  • .discrete(seconds:) - Debounces parameter updates for a specified time interval before executing them. swift // Sets the delivery mode to discrete with a debounce interval of 200ms. parameter.deliveryMode = .discrete(seconds: 0.2)

Defining an Applicator

An applicator can be defined and added using the MimiProcessingParameter.applicator() method. Since the applicator is held weakly by the parameter, you will need to maintain a strong reference to it.

// Make sure to hold on to the applicator strongly.
let applicator = parameter.applicator()
    .canApply { value in
        // Verify if value can be applied

        return true // or false
    }
    .apply { value, result in
        // Apply the value and execute result handler
        result(.success) // or result(.failure(error))
    }

Alternatively, an applicator can also be created while synchronizing an existing value to the parameter. This method returns a tuple containing the newly registered applicator and a Publisher that wraps the application task for applying the synchronized value.

// Make sure to hold on to the applicator strongly.
var (applicator, resultPublisher) = parameter.applicator(synchronizing: aValue)
applicator = applicator
    .canApply { value in
        // Verify if value can be applied

        return true // or false
    }
    .apply { value, result in
        // Apply the value and execute result handler
        result(.success) // or result(.failure(error))
    }

cancellable = resultPublisher
    .sink { completion in
        // Do something with completion
    } receiveValue: { value in
        // Do something with value
    }

Application timeout

It is also possible to set a timeout for the application of a value on the MimiProcessingApplicator. If the value application exceeds this timeout, it will cause the entire value application sequence to fail.

// Make sure to hold on to the applicator strongly.
let applicator = parameter.applicator()
    .timeout(0.5) // A timeout of 500 milliseconds
    .canApply { value in
        // Verify if value can be applied

        return true // or false
    }
    .apply { value, result in
        // Apply the value and execute result handler
        // If the timeout expires before the result handler is executed
        // the value application will be considered a failure.
        result(.success) // or result(.failure(error))
    }

Removing an Applicator

Since the applicator is held weakly by its associated parameter, simply setting the applicator to nil is enough to remove it from the parameter’s list of applicators.

Session Interruptions

The processing session provides the ability to be interrupted via the MimiProcessingSession.interrupt(reason:) method. Interrupting a session leads to its isEnabled parameter to be set to false. This value cannot be changed until all the session interruptions are resolved via the MimiProcessingSession.resolve(interruption:) method. Once all the interruptions have been resolved, the value of the isEnabled parameter is set back to its pre-interruption value.

// Interrupting a session
let interruption = "MySessionInterruption"

cancellable = session.interrupt(reason: interruption)
    .sink { completion in
       // Do something with completion
    } receiveValue: { isInterrupted in
        // Do something with isInterrupted
    }

// Resolving session interruption
cancellable = session.resolve(interruption: interruption)
    .sink { completion in
       // Do something with completion
    } receiveValue: { isInterrupted in
        // Do something with isInterrupted
    }

Note: Certain events in MimiSDK such as launching of the test flow also lead to the session being temporarily interrupted.

Parameter Recommendations

Based on certain events in MimiSDK e.g. user authentication, hearing test submission etc, the processing parameters are automatically set to recommended values.