How We Created a VR Tour Using Samsung VR Framework

Virtual reality has gone from a sci-fi topic to part of what we expect from technology; virtual reality powers software that helps us study, play games, design and create beautiful things, and learn more about the world.

At Yalantis, we always invest time into research projects that rely on new technologies, which is why we decided to create our own VR tour.

VR tours are often used for educational and entertainment purposes. We created a simple VR tour just like those that are often used to introduce world-famous museums.

To work with virtual reality, developers need to choose

At Yalantis, we work with Samsung Gear and the Samsung Galaxy S6 Edge.

Development tools that are commonly used for virtual reality apps include:

  • Unity 3D – a game development engine
  • Unreal Engine – another tool for game development
  • Oculus Mobile SDK – a set of native components written for VR app development

Gear VR Framework – a Java wrapper for the Oculus Mobile SDK and Google VR. This wrapper has a useful API and is well-documented; the best thing about this framework is that it allows us to develop applications for both Samsung Gear and Daydream at the same time.

Virtual tour development Android Samsung GearVR

This schematic shows the architecture of Samsung’s GearVR Framework. When we talk about development for Samsung devices, it all starts with the following files: 

backend_oculus_debug.aar,  framework_debug.aar 

If your goal is to develop an application for Daydream, you’ll need the following files:

backend_daydream_debug.aar

framework_debug.aar

All these files can be downloaded from GitHubAfter you’ve downloaded the files, you need to create a folder called libs and move them into it.

Then we add a couple lines of code to our app build.gradle

repositories {
   maven { url 'https://maven.fabric.io/public' }
   maven {
       url "https://oss.sonatype.org/content/repositories/snapshots/"
   }
   jcenter()
   flatDir {
       dirs 'libs'
   }
}

Next, we update our dependencies block:

ext.gearvrfVersion = '3.1.1-SNAPSHOT'
project.ext.jomlVersion = "1.9.1-SNAPSHOT"

dependencies {
   compile(name: 'backend_oculus-debug', ext: 'aar')
   compile(name: 'framework-debug', ext: 'aar')
   compile "org.joml:joml-android:${jomlVersion}"
}

We also need to add the Java OpenGL Math Library to our project. After all, libraries are added and before we start the development itself, we need to sign our application with the help of the Oculus Signature file (*.osig)This is necessary so we can launch our application without the head-mounted display. The Oculus Signature file (*.osig) can be generated with the help of the Osig generator.

After this file is generated, you can enable development mode:

Settings → Applications → Application Manager → Gear VR Service → Storage → Manage Storage → Enable Developer Mode.

This is when the actual development starts. To make your app available for the VR device, you need to add the following code to AndroidManifest.xml:

<meta-data           android:name="com.samsung.android.vr.application.mode" android:value="vr_only"/>

As with any other Android application, you will need a launcher activity. It’s important to remember that our Activity has to inherit the GVRActivity class since this class contains all business logic that enables the proper functioning of the VR headset.

In general, our activity serves as a point of entrance to the app, whereas most important things happen inside other components.

 

VR application architecture

 

[VR application architecture]

This diagram shows all the main components of the VR application architecture.

One level below these elements is GVRMain, which I would call the scene controller. GVRMain is responsible for switching from one scene to another and deleting them from the stack.

GVRMain has its own lifecycle, which is described with the following methods: 

onEarlyInit() 

​onAfterInit() 

​

The first one if the first method of the lifecycle, the second one is called at the end of the lifecycle. There’s also the onStep method that’s being called all the time. Usually, this helps track the number of frames per second.

Our implementation of GVRMain will contain scenes, so we need to describe them. A Scene or GVRScene is a component that provides a user interface. In the GVR Framework, there are a few of already implemented scenes that represent various objects:

  • GVRCubeSceneObject – a cube
  • GVRSphereSceneObject – a sphere
  • GVRCylinderSceneObject – a cylinder
  • GVRConeSceneObject – a cone
  • GVRViewSceneObject – a scene that can contain UI elements of an Android app
  • GVRCameraSceneObject – a scene that allows video streaming from a phone’s camera

As you can see, these scenes are not enough to create a full-scale mobile app.

We decided to create our own VR app using elements of the architecture described above. This app will be used for demonstrating virtual reality tours of various buildings and locations. The app will display simple three-dimensional scenes with virtual markers that indicate how to move to the next scene.

We can start developing a VR app like this by aligning sets of data for our rooms that will be displayed in the app as well as for the markers.

I created two models: Room and Marker. Let’s take a closer look at the Marker model:

​
public class Marker {

  private float x;

  private float y;

  private float z;

  private float rotationDegrees;

  private String room;

  public Marker(float x, float y, float z, float rotationDegrees, String room) {

      this.x = x;

      this.y = y;

      this.z = z;

      this.rotationDegrees = rotationDegrees;

      this.room = room;
  }

  public float getX() {

      return x;

​

 


  }
  public float getY() {
      return y;

  }

  public float getZ() {
      return z;

  }

  public String getRoom() {
      return room;

  }

  public float getRotationDegrees() {

      return rotationDegrees;
  }

}

The Marker model includes x, y, and z coordinates, as well as an angle at which a mark will be turned in three-dimensional space and the name of the room this marker will open.

The Room model is relatively simple:

​
public class Room {

  @DrawableRes
  private int textureFuture;
  private List<Marker> markers;
  public Room(@DrawableRes int textureFuture, List<Marker> markers) {
      this.textureFuture = textureFuture;
      this.markers = markers;
  }
  public int getTextureFuture() {
      return textureFuture;
  }
  public List<Marker> getGVRSceneObjects() {
      return markers;
  }
}

​
It’s obvious that we need a separate class to initialize our data. We didn’t use any database, and initialization happens when the app is launched. For a small sample project like we did this wasn’t critical, but if you’re considering a large commercial VR project, it would require more thorough development of the data structure.The Room model only has two fields: a link to the resource (a spherical image of the room in our case) and a list of markers for the room.

Here’s what our solution looked like: 

public class RoomsInitializer {

   public static final String ANDROID = "Android";
   public static final String BALCONY = "Balcony";
   public static final String BAR_ONE = "bar_one";
   
   private Map<String, Room> mRoomMap = new HashMap<>();

   public void init() {
       initRoom(ANDROID);
       initRoom(BALCONY);
       initRoom(BAR_ONE);
   }
   
   private void initRoom(String roomName) {
       createMarkers(roomName);
       Room room = new Room(getTexturePathByName(roomName), createMarkers(roomName));
       mRoomMap.put(roomName, room);
   }
   
   private List<Marker> createMarkers(String roomName) {
       List<Marker> markers = new ArrayList<>();
       switch (roomName) {
           case ANDROID:
               markers.add(new Marker(-1.0f, -0.1f, -0.1f, 90f, BALCONY));
               markers.add(new Marker(1f, 0.0f, -0.23f, -60f, BAR_ONE));
               break;
           case BALCONY:
               markers.add(new Marker(0.6f, 0.0f, -0.1f, -60f, ANDROID));
               break;
           case BAR_ONE:
               markers.add(new Marker(0.6f, -0.4f, -0.1f, -60f, ANDROID));
               break;
        }
    }
    
    @DrawableRes private int getTexturePathByName(String roomName) {
       switch (roomName) {
           case ANDROID:
               return R.drawable.android_room;
           case BALCONY:
               return R.drawable.balcony;
           case BAR_ONE:
               return R.drawable.bar_one;
       }
       return 0;
    }
    
    public Room getRoomByName(String roomName) {
       return mRoomMap.get(roomName);
    }
   
 }


We decided that our VR tour would only include three rooms. To work with the data, we needed to create a universal scene that would represent any of these rooms. Let’s take a look at how this class functions.

public class RoomScene extends GVRScene {


  private boolean isAnimating;

  private GVRPicker mPicker;

  private GVRSceneObject mEnviroment;

  private List<GVRSceneObject> markers = new ArrayList<>();

  private Room mRoom;

  public RoomScene(GVRContext gvrContext) {

      super(gvrContext);

      mRoom = App.getRoomsInitializer().getRoomByName(RoomsInitializer.ELEVATOR);

      addSceneObject(mEnviroment = makeEnvironment(gvrContext));

      makeMarkers(gvrContext);

  }
  private GVRSceneObject makeEnvironment(GVRContext context) {

      Future<GVRTexture> tex = context.loadFutureTexture(new GVRAndroidResource(context, mRoom.getTextureFuture()));

      GVRSceneObject environment = new GVRSphereSceneObject(context, false, tex);

      GVRSceneObject headTracker = new GVRSceneObject(context,

              context.createQuad(0.1f, 0.1f),

              context.loadTexture(new GVRAndroidResource(context, R.raw.headtrackingpointer)));

      headTracker.getTransform().setPosition(0.0f, 0.0f, -1.0f);

      headTracker.getRenderData().setDepthTest(false);

      headTracker.getRenderData().setRenderingOrder(100000);

      environment.getRenderData().setRenderingOrder(1);

      getMainCameraRig().addChildObject(headTracker);

      return environment;
  }
  private void makeMarkers(GVRContext context) {

      Future<GVRTexture> texture = context.getAssetLoader().

              loadFutureTexture(new GVRAndroidResource(context, R.drawable.activator_hotspot_button));

      GVRTexture gvrTexture = null;

      try {

          gvrTexture = texture.get();

      } catch (InterruptedException | ExecutionException e) {

          e.printStackTrace();
      }

      PickHandler pickHandler = new PickHandler();

      getEventReceiver().addListener(pickHandler);

      mPicker = new GVRPicker(context, this);

      removeMarkers();

      for (Marker marker : mRoom.getGVRSceneObjects()) {

          GVRSceneObject sphereObject = new GVRSceneObject(context, 100, 100, gvrTexture);

          GVRSphereCollider collider = new GVRSphereCollider(context);

          collider.setRadius(100);

          sphereObject.attachComponent(collider);

          sphereObject.getTransform().setScale(0.001f, 0.001f, 0.001f);

          sphereObject.getTransform().setPosition(marker.getX(), marker.getY(), marker.getZ());

          sphereObject.getTransform().rotateByAxis(marker.getRotationDegrees(), 0, 1, 0);

          sphereObject.getRenderData().setRenderingOrder(GVRRenderData.GVRRenderingOrder.OVERLAY);

          sphereObject.setPickingEnabled(true);

          sphereObject.getRenderData().setDepthTest(false);

          sphereObject.setName(marker.getRoom());

          markers.add(sphereObject);

          addSceneObject(sphereObject);

      }

  }


   private void removeMarkers() {

      for (GVRSceneObject object : markers) {

          removeSceneObject(object);
      }
  }
  private class PickHandler implements IPickEvents {

      private GVRSceneObject pickedObject;

      public void onEnter(GVRSceneObject sceneObj, GVRPicker.GVRPickedObject pickInfo) {

      }

      public void onExit(GVRSceneObject sceneObj) {

      }

      public void onNoPick(GVRPicker picker) {

          pickedObject = null;

          isAnimating = false;

      }

      public void onPick(GVRPicker picker) {

          GVRPicker.GVRPickedObject picked = picker.getPicked()[0];

          pickedObject = picked.hitObject;

          if (!isAnimating) {

              isAnimating = true;

              new GVRRotationByAxisAnimation(pickedObject, 1f, 720, 0, 1, 0)

                      .start(getGVRContext().getAnimationEngine())

                      .setOnFinish(new GVROnFinish() {

                          @Override

                          public void finished(GVRAnimation gvrAnimation) {

                              isAnimating = false;

                              if (pickedObject != null) {

                                  mRoom = App.getRoomsInitializer().getRoomByName(pickedObject.getName());

                                  Future<GVRTexture> textureFuture =

                                          getGVRContext().loadFutureTexture(new GVRAndroidResource(getGVRContext(), mRoom.getTextureFuture()));

                                  makeMarkers(getGVRContext());

                                  try {

                                      mEnviroment.getRenderData().getMaterial().setMainTexture(textureFuture.get());

                                  } catch (InterruptedException | ExecutionException e) {

                                      e.printStackTrace();

                                  }

                              }

                          }

                      });

          }

      }

      public void onInside(GVRSceneObject sceneObj, GVRPicker.GVRPickedObject pickInfo) {

      }
  }

}

First of all, we’ll start with initializing the space. In essence, space here is the GVRSceneObject on which we overlay our texture.

We also need to add a cursor to the makeEnviroment method since that will let the method interact with the environment. The cursor is also a GVRSceneObject; and once all objects are initialized, we can add them to the scene.

For our VR tour to work, we need to add markers that will reload our scene to display a particular room.

To do this, we need the makeMarkers method. Each marker is also a GVRSceneObject with a permanent texture. When your cursor hovers over an object, a listener appears.

We implement this logic in the onPick method. This is where we animate the marker and reload the scene for a particular room. As a result, we have a virtual tour of our office.

 
4.7/ 5.0
Article rating
13
Reviews
Remember those Facebook reactions? Well, we aren't Facebook but we love reactions too. They can give us valuable insights on how to improve what we're doing. Would you tell us how you feel about this article?