Scene Class

Owned by Connor Montgomery (Deactivated)
Last updated: Sept 06, 2022 by Chitra Borker (Deactivated)

This section describes the different settings that you can configure for the Scene Class. 

Constructor

The ​Scene Class​ is instantiated with the new operator, and returns an instance of a Scene as follows:

new Scene( videoElement?: HTMLElement, audioOnly?: boolean, requestedUserMedia?: UserMedia, requiredUserMedia?: UserMedia ): Scene

 Example Scene Instantiation:

var smscene = new Scene(element);

 Parameters

  • optional​ ​videoElement​: ​HTMLElement, a reference to the desired HTML element where​ the Video output should be displayed. ​Mozilla developer documentation​ has more information on the JavaScript type interface for ​HTMLElement​.

  • optional​ audioElement​​: ​boolean​, in cases where very low bandwidth can be detected, it may be desirable to offer an audio only experience by setting this argument to ​true​.

  • optional​ ​requestedUserMedia:​ UserMedia​, by default, the Soul Machines Web SDK will​ request both Microphone and Webcam access. Optionally, one of three values of the UserMedia​ enumeration can be supplied in this argument to specify otherwise.

UserMedia.None UserMedia.Microphone UserMedia.MicrophoneAndCamera (default)

  •  optional​ ​requiredUserMedia​: ​UserMedia​ by default, the Soul Machines Web SDK will require neither the Microphone nor the Camera, and the connection will still succeed even if the User does not grant access. The end User will still see and hear the Digital Person and will be able to communicate with the Digital Person via text entry.
    If fewer user media devices are requested than are required, then the requirements take precedence. If the user media requirements are not met, then ​Connect()​ will fail. The same three enumeration values of UserMedia​ are available as possible values for this argument.

 Returns: Scene​

Accessors

An instance of the Scene class provides a selection of ​getters​ and ​setters​ that provide convenient access to instance properties, and enable the ability to assign event listener functions.

Event Listener Getters and Setters

An instance of the Scene class provides a set of accessors to listen for certain events of interest.

setter​ ​onConversationResultEvents(): PersonaEventMap

Returns an object, indexed by strings, of instances of ​SmEvent​.

 Refer to the ​Soul Machines Event Class section for the definition of SmEvent.          

setter ​ onDisconnected(onDisconnected: Function(SmEvent){}): void​                           

Setter that assigns an event listener that is invoked when the RTC connection to the Digital Person is ended. This function returns ​void​. The assigned event listener is invoked with an instance of SmEvent.  ​

getter ​ ​onDisconnectedEvent(): SmEvent

A getter that returns the value of the SmEvent instance specific to a disconnect event.

setter ​ ​onRecognizeResults(onRecognizeResults: Function(SmEvent){}): void

Setter that assigns an event listener that is invoked when results of a given query are recognized. This function returns ​void​. The assigned event listener is invoked with an instance of SmEvent.

getter ​ ​onRecognizeResultsEvent(): SmEvent

A getter that returns the value of the SmEvent instance specific to a disconnect event.

setter ​ ​onSpeechMarkerEvents(onSpeechMarkerEvents: Function(SmEvent){}): void

Setter that assigns an event listener that is invoked when a Speech Marker is recognized within the conversational content. For example, a speech marker could be utilized to display a certain interface element within the UI. This function returns ​void​. The assigned event listener will be invoked with an instance of SmEvent. ​

setter ​ ​onState(onState: Function(SmEvent){}): void

Setter that assigns an event listener that is invoked when the state of the Scene changes. This function returns ​void​. The assigned event listener will be invoked with an instance of SmEvent.

getter ​ ​onStateEvent(): SmEvent

A getter that returns the value of the SmEvent instance specific to a disconnect event. ​

setter ​ ​onUserText(onUserText: Function(SmEvent){}): void

Setter that assigns an event listener that is invoked when the end user enters text input in the session. This function returns ​void.​ The assigned event listener will be invoked with an instance of SmEvent. ​

getter ​ ​onUserTextEvent(): SmEvent

A getter that returns the value of the SmEvent instance specific to a user text event. ​

 

Other Accessors

getter ​ ​videoElement(): HTMLElement | undefined

Returns ​HTMLElement​ or ​undefined​. ​Mozilla developer documentation​ has more information on the JavaScript type interface for ​HTMLElement​.

 

Instance Methods

configure(configuration: ConfigurationModel): Promise<any>

Once an instance of the ​Scene​ class is instantiated, several properties such as audio squelch levels and lighting, can be configured by invoking the ​configure()​ instance method. The specific attributes available for configuration are available as defined in the ​ConfigurationModel​ class.

Refer to the ​Configuration Model Class section for the definition and details of the available properties.

connect(serverUri: string, userText: string, accessToken: string): Promise<any>

Your Soul Machines Solution Architect provides you with a Server URI and Public Access Token (JWT Token) for your Development and Production Digital Person. This method is invoked to connect to the Digital Person. 

Error Messages:
The error result has two fields: “message” which is the message string and “name” which is one of the following error name/reason codes:

  • notSupported - The browser does not support getUserMedia.

  • noUserMedia - The user microphone and/or camera is not available.

  • serverConnectionFailed - The connection to the server failed.

  • noScene - No persona is available.

  • mediaStreamFailed - The audio/video stream failed.

  • sessionTimeout - The session timed out before it was fully available.

interface RetryOptions { maxRetries?: number; delayMS? number; }

This is a connection retry mechanism to ensure the system works reliably. The retry option is configurable, both with the number of attempts and the delay in between.
The "maxRetries" specifies the number of connection retries including the first connection attempt. The "delayMs" specifies the delays in milliseconds between each retry.  If not provided, default "maxRetries" is 50 and "delayMs" is 200. 

session.keepAlive(): number

This is an inactivity timeout reset mechanism to ensure the connection of the end user to the Digital Person is kept alive, even if there is no conversation exchanged.

The inactivity timeout configuration setting enables the session of the end user with the Digital Person to be automatically closed, if the conversation becomes idle and the Digital Person does not speak for a period of time (typically this is 5 minutes, but can be configured differently based on customer request).

The "keepAlive" method resets the inactivity timeout timer so after it has been called, the connection will not timeout for another 5 minutes (or whatever has been configured).

connectionValid(): boolean

This method can be invoked at any time to determine if the connection to the Digital Person is valid, it does not necessarily mean it is currently connected. To determine if the Scene is connected, use isConnected() defined below. Simply returns true or false.

disconnect(): void

Invoked to disconnect from the Digital Person, ends the conversation session. Returns void.

getState(): Promise<any>

Returns a Promise that resolves with an object representing the current state of the Scene instance with all attributes.

isCameraConnected(): boolean

Can be invoked to check if the end user’s camera is connected, and returns a Boolean that is true​ if the​ camera is enabled and connected, otherwise returns false.​

isConnected(): boolean

Can be used to check if the Scene is valid and​ connected. Returns true​ if Scene is both valid and connected.

isMicrophoneConnected(): boolean

Can be invoked to check if the end user’s microphone is connected, returns a Boolean that is true if the microphone is enabled and connected, otherwise returns false​.

sendVideoBounds(width: number, height: number): void

In the event that the HTML element presenting the video is resized, this method can be used to reset the bounds of the Scene video. This gives the application the chance to choose what size should be rendered on the server and the application is responsible to register for a video element size change event and call this method to maintain best possible video quality for the size and/or to set an updated video element size and then call this method. This method returns void.

Parameters

  • width​: integer. The width in pixels to render the video

  • height​: integer. The height in pixels to render the video

 

startRecognize(audioSource?: AudioSourceTypes): Promise<any>

Invoking this method starts the Speech-to-Text (STT). Your Soul Machines Solution Architect will work with you to define your selected Speech-to-Text engine, such as Google Speech API.
This method starts the specified speech recognizer. The method returns a Promise that will succeed once Speech recognition has begun.

There is one optional Parameter, ​audioSource​. This is defined in ​AudioSourceTypes​ within the SDK, and is one of the following:

  • smwebsdk.audioSource.processed

  • smwebsdk.audioSource.squelched

The value of this parameter defaults to ​smwebsdk.audioSource.processed​

stopRecognize(): Promise<any>

Invoking this method stops the Speech-to-Text recognition.