Magic Windows - Working with AR Kit

Setting up ARKit in unity


There is no plugin at the asset store! So go here:

The zipped file looks like a Unity Package, but it isn’t! You should copy its content to your project “Assets” folder.

Unity will update

Unlike Vuforia, Arkit doesn’t create a new Camera object, but uses Unity’s

·         Create an empty object named “Camera Parent”

·         Move the “Main Camera” to “Camera Parent” – making it ‘child’.

o   Main Camera – Inspector

§  Clear Flags: Depth Only

§  Culing Mask: Everything

§  Field of View: 60

§  Clipping Planes: Near: 0.1  / Far: 37.81

o   Add component: Unity AR Video

§  Material: YUVMaterial (or any clear material)

o   Add component: Unity AR Camera Near Far

·         Create empty object: ARCameraManager

o   Add component: Unity AR Camera Manager

§  Camera: Drag n drop the Main Camera

§  Plane Detection: Select if you want it to track planes on either horizontal or vertical or both. Sometimes you don’t want to track walls or the floor.

§  Get Point Cloud: toggle

§  Enable Light Estimator: Toggle

§  Enable Auto Focus: Toggle (YES!)

§  DONE!

ARKit is setup. Now setup the example scene

·         Create Empty Object “DebugViz”

o   Inside it create another empty object named: PointFinder

§  Add component: Unity Point Cloud Example

·         Increase Num Points to Show

·         Point Cloud Prefab: (grab example asset) PointCloudPrefab

§  Add another component to DebugViz: PlaneFinder

·         Add component: Unity AR Generate Plane

o   Plane Prefab: debugPlanePrefab (also part of the example collection)

Build for IOS

·         Player Settings

o   Bundle Identifier: add your company name:

o   Camera Usage Description: (Type whater you want to the user allow the use of the phone’s camera…..could be “ARBABY”…whatever.

o   Target minimum iOS Version:  11.0 (it has to be 11.0 or above)

o   Requires ARKit support: toggle

Open the project in XCode and go through all the usual Apple nuisance of adding team/your apple developer coder (automatic); connect a phone to your computer; authorize everything and if everything works you should have an icon with the Unity App in your phone.


Unity AR Hit teste example

·         Create empty object: hit_parent

o   Inside it create empty object: hit / OR…. Create a 3D object (exp: Cube) and ..

o   Add the “Unity AR Hit test example” script


§  Open the script

This script allows to detect if a touch on the screen is hitting one of the AR objects.

                Remove all the Update Unity part – from “#if” all the way to “else” and remove “endElse”. We don’t need it to run inside the unity player


                Remove comments for all ARHitTestResultType.xxxxxxx (activating them)


Build/ Compile and Run….it sort of works. A cube showed up and it stays in there…once. But no interaction.



Desert of the Real - Create a VR paintbrush

This is a 3-week-old assignment that I didn’t manage to do until now. The goal is to create a VR painting program.

Everytime I start to code I just freeze looking at the empty script. I just don’t know how to organize my thoughts and don’t have a clear idea of what is to be achieved. For this reason, I decided to write everything I want to do in plain English.

Before coding I set up the scene using the basic SteamVr package. I set up a Teleport area for locomotion; a plane to make this area visible; a cube platform to set my objects and for orientation; three cubes prefabs, each with a different material/color (red, green and blue) and finally a SteamVR player controller. I also put some throwable objects just to test the controller responsiveness.

The class template has an empty object called “InteractiveObject” where the BrushHandler and ColorSelector scripts are kept. The ControllerHandler script is found in the Players “RighHand”.




Now for the coding part:


·       Have a Brush object

·       The brush is controlled with the VR controller.

·       Have colored objects on the scene that will serve as colorPalette.

·       When Brush touches one of these objects – it changes to the objects color.

·       When pressing the Trigger Button on the controller, the Brush creates many copies of itself named paintObjects, allowing to paint the scene with it.

·       The paintObject must have the color of the brush.

·       If the brush touches a colorPalette with a different color, it should change color but the paintObject already on the scene should remain with its original color – Once paint is set, it should remain the same color. (no changing the color of what was already drawn)

·       There should be a limit on how many paintObjects are on the scene at any given time:  a limit on how much paint can be on the scene. If this limit is reached, the ‘oldest’ paintObject must be deleted.



There are three scripts already created by the class instructor Igal Nassima. They come with comments and guidelines, but I don’t really understand his workflow. I’m not sure where Igal want us to set the scripts; if the Brush object is created by the script or if it will already be on screen.

I will try to do it somehow…

·       ControllerHandler

o   Makes the brush paint

o   Checks what color is in the brush

o   Get the brush color

o   Creates instances of the paintObject that has the current color of the brush.

·       ColorSelector

o   “Create a public Color Type that can be set from the inspector to be used in each menu Item” – (comment by Igal) – Don’t understand what he means by “color type” and why it must be set from the inspector.

o   “Setup Collider Trigger to send the color of this object to the Brush Handler” – Checks if there is a collision between the brush and another object and send the color of this other object back to the Brush Handler.

o   “Find Object of Type à then call the function that sets the currentColor” –

·       BrushHandler

o   Store the current color on the brush

o   Sets what gameobject will be drawn/created/instantiated when activated by the trigger on the controllerHandler.

o      //create a public function that SETS UP the color of the currentColor (hint, function must have a Color variable)

o       //create a public function that RETURNS the Color of the currentColor

§  Don’t know the difference between Set UP the color and “Returns the Color” and what it is supposed to do..

§  Also, Don’t know how to make scripts communicate with one another.

Found an example at my colleague Ellen Nickles documentation of this assignment:

//Inside the ControllerHandler script it calls the the color on BrushHandler:
  private void OnTriggerEnter(Collider other)


        FindObjectOfType<BrushHandler>(). currentColor = myColor;





Creating the Brush

The Brush doesn’t paint, but stores its own color and changes color when

·       The brush is a Sphere prefab named ‘Brush’.

·       It will have a color.

·       It will get the color of the object it touches

·       It will return the color it has

·       It has a script called BrushHandler. It will set the following behaviors:

o   Stores the currentColor

o   Creates/Returns the prefab that will be drawn (the “paint”)


Starting to write the code itself

              Don’t really know where to start. I look at the three template scripts, but don’t know which function to make. I don’t even know if I should create the brush object of if it is instantiated by one of the scripts.


Instantiation Tests

In order to practice basic coding, I also created a Test script to try things out.

I created a transform function that makes a cube move and it asks for a GameObject. I added a different object on the inspector but the object where the script was attached – the cube -  still was the one that moved. Why?

1 Creating – Instantiating an object

              I will create a brush and make it instantiate copies of itself when user press the Oculus Controller Trigger button. I’ll figure the other actions (changing color) later.

              We managed to run a basic instantiate code copied from the Unity API manual. At first it didn’t worked, because we added the script to the prefab we wanted to instantiate. This freezes Unity because it was trying to instantiate the object WITH the instantiate script. Once we added a different prefab to the public GameObject slot in the Inspector the instantiate code worked fine.

Basic Unity Instantiate example 1:

//The Manual didn’t used “public”.  

 public Transform prefab;

    void Start()


        for (int i = 0; i < 30; i++)


            Instantiate(prefab, new Vector3(i * 2.0F,i* 1, 0), Quaternion.identity);



Basic Unity Instantiate example 2:

public class TestScript : MonoBehaviour


    // Instantiate a rigidbody then set the velocity


    public Rigidbody projectile;


    void Update()


        // Ctrl was pressed, launch a projectile

        if (Input.GetButtonDown("Fire1"))


            // Instantiate the projectile at the position and rotation of this transform

            Rigidbody clone;

            clone = Instantiate(projectile, transform.position, transform.rotation);


            // Give the cloned object an initial velocity along the current

            // object's Z axis

            clone.velocity = transform.TransformDirection(Vector3.forward * 10);






                  For some reason it seems that SteamVr plugin is no longer functional on my project. I can’t find the Steam Controller Input inside the ‘Window’ menu. I downloaded the template project again and it seems to be working fine there. I’ll have to setup my scene again.

Update: Maybe the SteamVR plugin wasn’t turned on (for some reason). I started a new project anyway.




Spheres everywhere! Instantianting is a success (and out of control)

Spheres everywhere! Instantianting is a success (and out of control)

Making the paint brush:

Now we have two scripts that allows us to create spheres on the scene. One creates on sphere at every frame and other creates one sphere when “Fire1”/ Ctrl is pressed on the keyboard.

By changing the prefab instantiated by theses scripts it was possible to create a static sphere on the scene – one that has no gravity or doesn’t fly away. By pressing “Fire1”/Ctrl it was possible to create flying sphere and ‘paint’ the scene with spheres.

Goal:  Create static instances of a sphere while the Oculus Controller Trigger button is pressed and stop creating when not pressing.

`This is the code created

using System.Collections;

using System.Collections.Generic;

using UnityEngine;

public class DrawTest : MonoBehaviour


public Transform paint;

void Update()

    if (Input.GetButtonDown("Fire1"))



void createPaint()


        for (int i = 0; i < 1; i++)

            Instantiate(paint, transform.position, transform.rotation);





This script works like the first one: It creates one sphere every time “Fire1”/Ctrl is pressed, but it doesn’t keep creating more balls if the button is still pressed. Maybe I need to change “Input.GetButtonDown” to something else…

Changed from “Input.GetButtonDown” – that returns true when the button is pressed down and only resets if released – to “Input.GetButton” – that returns true every frame the button is pressed. And it works!

Nice smooth sphere paint created on the scene.

Nice smooth sphere paint created on the scene.

practicing coding with P5 and coding train

Decided to revisit Dan Shiffmans Coding Train Videos from the very beginning to practice some basic JavaScript programming. Following tutorials are a good way to learn, but only by doing my own projects and solving the problems I need to solve will I really improve.

A few weeks ago I've met Jabril at ITP. He is a self-taught programmer with an amazing youtube channel, and of the many amazing insights he gave me was how much time he missed by doing a bunch of tutorials instead of having a more project base learning process. Since I'm starting from scratch and I like games, I decided to use the video tutorials with the specific goal of making my own Break Out game in P5.