ISAR SDK – Quickstart Guide for Android

ISAR SDK – Quickstart Guide for Android

Hey there!

In this quick start guide for ISAR we will show you how to easily set up your first ISAR project in Unity. For this we will need Unity Hub 3.0.1, Unity 2020.3.30, MRTK 2.7.3 and of course ISAR 2.4.0.0.

First, we check whether your system complies with the requirements necessary for an successful use of ISAR and install the necessary tools. Once this is done, we implement the ISAR SDK in Unity’s development environment, connect to the client device (in this guide, Android devices) and set up a demo scene via Microsoft's Mixed Reality Toolkit.

The first chapter will cover where to get everything required to get the ball rolling.

Chapter 1 – Preparations

Requirements for ISAR SDK

First, we should check if your system is good-to go. In the table below are listed the Hardware Requirements as well as the Networking Requirements for ISAR.

 

Minimum

Recommended

Operating System

Windows 10 (10.0..17763 Build 17763)

Windows 11

Windows Server 2019

Windows 10 (10.0..17763 Build 17763)

Windows 11

Windows Server 2019

Memory

16 GB

64 GB

CPU

Intel i5 8th Gen. 6 Cores

AMD Ryzen 7 3700X

Intel i7 12 Gen. 12 Cores

AMD Ryzen 9 3900X

GPU

NVIDIA GTX 1070Ti

NVIDIA GRID for VMs

NVIDIA RTX 3080 TI

NVIDIA GRID for VMs

Storage

SSD or NVMe

SSD or NVMe

Networking

WiFi 5GHz

Bandwidth 20 MBit

WiFi 5GHz

Bandwidth 40 MBit

Enable incoming TCP Port 9999

Enable outgoing UDP Ports 16384-32768

The local network of the AR device must be capable of using STUN (Session Traversal Utilities for NAT) in case the Server is not located in the same network as the Client. 

Your PC hardware specs can be checked by clicking on the Windows Start button and then navigating to Settings. In the Settings menu, press on System and then scroll down to About. There you will see specs for processor, Memory (RAM) and some other system info.

Please note that the Networking Requirements are not optional. ISAR only works properly if the necessary bandwidth is available, and the ports specified are open. Also make sure that you have enough rights to install third-party software.

Downloading and Installing Unity Hub

Unity is a cross-platform game engine developed by Unity Technologies. It is primarily employed to create three-dimensional (3D) and two-dimensional (2D) interactive simulations and applications. We use this development environment to implement the ISAR SDK in a project and build an application powered by remote-rendering.
If you already happen to have a version of Unity installed, you can skip this step.

The best way to employ Unity and launch a project is with Unity Hub. It is a tool used for managing multiple installations of the Unity Editor, creating new projects, and accessing already created projects.

Unity Hub can be found here.

Click on “Download Unity Hub” and run the executable file “UnityHubSetup.exe” which will be downloaded to your workstation. Confirm the Access Control and License Agreement and complete the installation process by clicking on “Install”. The program will be installed to the directory specified in Destination Folder. Once the setup is finalized, you can exit the installation wizard by clicking "Finish" when the window displays the message "Completing Unity Hub Setup". When Unity Hub is now launched, we are prompted to install the Unity Editor, which already brings us to the next step.

Downloading and Installing Unity

Now we are going to follow through with the Unity Editor and install it by clicking – you guessed right – on “Install Unity Editor”. Accept once again Access Control and the installation can begin. Once the setup is finished, the installed version will be listed within Unity Hub in “Installs”. In this guide this would be Unity 2020.3.30.

In case you want to use a different version of Unity, you will have to navigate to the official Unity Download Archive here.

Look for your preferred Unity version and click on the green button reading “UnityHub” next to its name.








For this guide we favored Unity 2020.3.30. When you are prompted to add modules check “Microsoft Visual Studio Community 2019”, if you want to add your own scripts to the project, else do not check anything and click on “Install”.


Agree with the License Term, if you are adding Visual Studio 2019, and then press on “Install”. Once the setup is finished, the installed version will be listed within Unity Hub in “Installs”.


Downloading MRTK 

MRTK-Unity (“Mixed Reality Toolkit”, a project-driven by Microsoft) provides, amongst other things, a cross-platform input system and set of spatial interactions and UI for XR development.

For this guide, we are going to use MRTK 2.7.3. It is available for download here.

Make sure to download the foundation package as well as the example package. The example package is needed, if you want to experience the MRTK demo scene as covered in chapter 4. 


Downloading ISAR SDK
The ISAR SDK is available on Holo-Light's GitHub page and can be downloaded there. 

We differentiate between the Full version and the Trial version of ISAR. You can access the Full version only with a valid fee-based license. The Trial version is free and can be accessed here. There are no differences in the implementation of ISAR between these versions – therefore, this guide is valid for both.

The required ISAR packages are downloaded by clicking on the green button reading "Code" on GitHub and then selecting the option "Download ZIP". The downloaded ZIP folder should now be located in your “Downloads” directory. Unzip it and place the folder in your preferred destination.

From the next step on, we can finally get down to business and implement ISAR in a Unity project.


Chapter 2 – Starting a Unity Project and Importing ISAR

Setting up a New Unity Project in UnityHub

Launch UnityHub and click on “New Project”. In the bar at the top, we can select the desired Unity version by clicking on the arrow and searching for it in the list.  Next, go to “All Templates” and click on 3D (Core). Set the project name as well as location for the project file and click on “Create Project”. It will open the Unity Development Environment subsequently.

Importing MRTK

Now we must import the MRTK into our (still) empty Unity project.


To do this, we click in the top tab bar on Assets -> Import Package -> Custom Package... and as soon as the file explorer pops up, you can search for and select the MRTK 2.7.3. foundation package file. If you want to test a MRTK demo scene (covered in chapter 4 of this guide), you will also need to import the MRTK 2.7.3. example package file in the same manner. Once the pop-up window reading "Import Unity Package" appears, confirm via "Import".


In some versions of Unity, the "MRTK Project Configurator" window appears for the first time as soon as the import of the MRTK is complete. You can simply close this window, even if it appears again at a later point in the project (e.g. when you reopen the project).


If the import of the MRTK packages was successful, they should be listed under Project -> Assets as follows:
​​

Removing "Version Control"

The package “Version Control” must be removed in Unity so that ISAR can run without errors.  To do this, click in the top tab bar on Window -> Package Manager -> Version Control -> Remove.

Version Control is not included in every Unity version. If your Unity version comes without “Version Control”, great – then you don't have to do anything in this step!

If you skip this step and do not uninstall "Version Control", the ISAR tab will not be displayed in the top row, and you will therefore not be able to use ISAR. We should keep the Package Manager tab open, as we will need it in the next step.

Importing ISAR 

This is where the star of the show comes into play: the ISAR SDK. Via Package Manager -> +-Sign -> Add package from disk you can navigate to the folder previously downloaded from GitHub. We are looking for the ISAR package files so the path for the Trial Version would be for example:
  1. {PATH} \ISAR-SDK-Trial-main\ISAR-SDK-Trial-main\Packages\com.hololight.isar
  2. {PATH} \ISAR-SDK-Trial-main\ISAR-SDK-Trial-main\Packages\com.hololight.mrtk

Within these two folders, the file "package.json" must be selected respectively. Please make sure that you import both packages for smooth use.



Afterwards the Package Manager should look like this:


The names of the Full version folders are analogous, and their packages are imported in the same manner.

Adjusting Project Settings

After the ISAR packages have been imported, the next step is to adjust the project settings.

To achieve this, we click in the top bar on Edit -> Project Settings -> XR Plug-in Management and tick the option ISAR XR within Plug-in Provider.

If this check mark is not set, the Unity project will crash when "Play" is pressed.

Running the ISAR Unity Project

First, we must configure the MRTK for ISAR – and this is simply done with one click via ISAR -> Configure MRTK. This step must be done for each new scene before the project is ready to run.

As a next step, we need to get the project up and running by either entering the play mode in the Unity Editor or building a standalone application.

The play mode is triggered in Unity by pressing the “Play” button. The connection to the server can now be established using one of the client applications. This step is described in chapter 3.

A standalone application can be built by navigating to File -> Build Settings -> PC, Max & Linux Standalone -> Build. If you have no object imported currently, you will get a black screen once you establish a connection. By clicking on “Add Open Scenes” you can import your scene(s) into the standalone application. This option is important, once you imported 3D objects into Unity.

If you prefer the standalone version, you should apply these settings:
  1. Platform: PC, Mac & Linux Standalone
  2. Target Platform: Windows
  3. Architecture: x86_64

Next, save the build in your preferred directory. Outside of Unity, navigate to the .exe file and run it as an administrator.



The build will be launched and show only a black screen – no worries, once you establish a successful connection as covered in chapter 3, your scene(s) should appear on your client device.

The ISAR Server application will in turn start listening on the TCP port 9999, which is needed to establish the streaming session. If the connection should fail, ensure that it is not being blocked by a firewall.

Disabling ISAR

Once successfully implemented, ISAR can be disabled while remaining as a package in the project. To do so, we first must click on Edit -> Project Settings -> XR Plug-In Management and uncheck the option ISAR XR within Plug-in Provider.

Check Unity Mock HMD and ensure not to call the ISAR class, or any of the inherited classes (IsarViewPose, IsarAudio, IsarCustomSend or IsarQr) constructor within the code.

In addition, disable the QrSupport script, if it happens to be enabled.


Chapter 3 – Getting the Clients ready

Once you have progressed this far, you have installed the necessary tools and implemented the ISAR SDK in a Unity project, great! But that's only half the fun: In this chapter, we will focus on the client applications that make the invisible visible.

The ISAR SDK currently supports HoloLens 2, Oculus Quest 2 and Android.

To establish a connection between the server (your workstation) and the preferred client application, all devices involved must be on the same network. This requirement cannot be evaded, as no connection can be setup in any other way.

Setting up the Configuration File

To ensure that our client application works flawlessly, we must first set the correct parameters in the configuration file within Unity. The configuration file can be accessed via the tab Project -> Assets -> Streaming Assets and is called "remote-config.cfg”.


Open it by right-clicking and selecting your preferred text editor. Adjust the parameters listed below for your client and then save the changes. The author of this guide has chosen Wordpad in this case.

The following parameters can – and should be changed – depending on the client device. Please note: Changes to the configuration file in “Streaming Assets” will be overwritten each run and each build; therefore, it is recommended not to make changes to the copied file.

  1. width and height determine the resolution of the rendered image per view/eye
    1. The resolution needs to be supported by the H264 hardware decoder of the client device
  2. numViews determines the number of views/eyes rendered on the client device
    1. Value 1 enables mono rendering, e.g. Tables
    2. Value 2 enables stereo rendering, e.g. HoloLens
  3. bandwidth allows for specifying the bitrate for the decoder
    1. If no value is set, a default value of 20mbit is used
The table below contains the required values for each client application.

Client

Width

Height

NumViews

Bandwidth

HoloLens 2

1440

936

2

36000

Oculus Quest 2

1920

1920

2

50000

Android

1920

1080

1

36000

Downloading and Installing the ISAR Client Application for Android

The ISAR Client application for Android can be downloaded from the Google Play Store. 

Start your Android device and search for "ISAR Client" in the Play Store. If you now click on the "Install" button, the app will be installed and should be among your installed apps.

Establishing a Connection between ISAR Client and Server

Start your ISAR project by pressing the “Play” button within the Unity editor. Launch the ISAR Client application on your Android device and enter the IP address of your server (e.g.) in the text field. You can obtain your server’s IP address by entering "cmd" in the Windows search bar and then typing "ipconfig" in the command window. The required IP address is listed under "IPv4 Address".


Click on “Connect” in the client application and once the connection is successfully established, you should be able to see your models on your Android device.

Known Issues

  1.  Unity HDRP Project: Rendering has dark grey artefacts close to light sources 
    2. Unity HDRP Project: Touch Input is not working

    Chapter 4 – Setting Up a MRTK Demo Scene

    Now that we have prepared server and client, we can experience the power of remote-rendering for ourselves. And for that we use a demo scene provided by the MRTK for a first sample.

    Loading a MRTK Demo Scene and Preparing the Scene

    We have already done the groundwork for this step with the import of the MRTK packages in chapter 2.

    If we navigate to Assets in the project tab and then click on Assets -> MRTK -> Examples -> Demos -> Input -> Scenes -> PointerResult, we find the ready-made scene we are looking for.


    On the left you will find the desired scene file (with the Unity icon), which you can open by double-clicking on it. A window will pop up called “TMP Importer”, confirm the action via clicking on “Import TMP Essentials” and close the pop-up window.


    In Unity’s scene viewer and hierarchy will now appear “PointerResultExample”.


    When the configuration file has been adjusted for the Android client application (see chapter 3), we will have to configure the scene by clicking in the top bar on “ISAR” and then on “Configure MRTK”. Once the scene has been prepared, the connection to the server can be established.

    The scene consists of an empty pinboard asset plus information panel, onto which post-its can be stuck by directly clicking on the pinboard displayed in the screen.










      • Related Articles

      • ISAR SDK – Quickstart Guide for HoloLens 2

        Hey there! In this quick start guide for ISAR we will show you how to easily set up your first ISAR project in Unity. For this we will need Unity Hub 3.0.1, Unity 2020.3.30, MRTK 2.7.3 and of course ISAR 2.4.0.0. First, we check whether your system ...

      • ISAR SDK – Quickstart Guide for Oculus Quest 2

        Hey there! In this quick start guide for ISAR we will show you how to easily set up your first ISAR project in Unity. For this we will need Unity Hub 3.0.1, Unity 2020.3.30, MRTK 2.7.3 and of course ISAR 2.4.0.0. First, we check whether your system ...

      • Installing and Running ISAR Client Application for Android

        To benefit from ISAR's remote rendering technology, it is necessary to employ a server running the ISAR SDK (e.g. a Unity project with integrated ISAR SDK) as well as an ISAR Client application. If you want to stream your projects on an Android ...

      • Troubleshooting – ISAR SDK and Client Applications

        No software is immune to bugs - not even ISAR, admittedly. We have compiled below a collection of possible solutions stemming from real use cases that will may help you to solve problems independently (without frustration). If your issue is not ...

      • Overview of ISAR Clients

        ISAR SDK currently supports several client devices (Microsoft HoloLens 2, Oculus Quest 2, Android) that can connect to an ISAR server application with the respective ISAR client application. To establish a successful connection between ISAR server ...