Tutorial: Kinect to Flash

In this tutorial, you will learn how to make Microsoft’s Kinect sensor communicate with an swf file as I recently demonstrated in a previous post. It actually doesn’t require a ton of code, but it does involve several different technologies and there is a bit of set up to get the Kinect side going that I won’t be going into detail on. However, links to the channel9 video tutorials are provided.

This tutorial will focus on the modifications to the C# code, the Node.js server and of course the Flash file and ActionScript. As usual, all files are available for download at the end of this tutorial.

First off, you are going to grab the Kinect for Windows SDK from Microsoft. You can visit the official site to get started. You will also want to have a version of Microsoft’s Visual C# compiler; the free, express edition is fine. Once you’ve successfully installed the SDK, I’d recommend heading to the Channel9 quick start guide and watching the following videos:

Installing and Using the Kinect Sensor
Setting Up Your Development Environment
Skeletal Tracking Fundamentals

The skeletal tracking demo is the one we will modify later on to send data to our server.

If you have made it through the Channel9 videos mentioned above, you’re ready to move on to the Node.js server. I recommend you take a look at my previous Node.js tutorial to get a handle on it if you don’t already have it installed.

Go ahead and open up a new blank text document and save it as KinectToFlash.js and enter the following code.

var net = require('net');
var mySocket;

var server = net.createServer(function(socket) {
	mySocket = socket;
	mySocket.on("connect", onConnect);
	mySocket.on("data", onData);
});

function onConnect()
{
	console.log("Connection Established");
}

function onData(d)
{
	if(d)
	{
		mySocket.write(d);
	}
}

server.listen(9001, "127.0.0.1");
console.log("Server running...");

What we've done here is set up a very simple server that listens for connections from the local machine. We have the server spit out some text, "Server running...", so that we can see it running in the console window. We also have it notify us when a new connection is established. Then, when the server receives data, it is sent back to the socket.

There is one thing here that should be noted. This is not a good server and it is flawed in many ways. I'm only using this because this is just a simple demo. If I were intending to do anything more with this I would at least be keeping track of the Kinect and Flash sockets individually. Future projects that I do will not use this server.

If you want to test your server, open the command console and navigate to the Node.js bin directory. Now type: node KinectToFlash and press the Enter key. You should see the "Server running..." text appear in the console window. Cool? Ok, press ctrl+c to kill it.

Pages: 1 2 3

Bookmark and Share

16 Responses to “Tutorial: Kinect to Flash”

  1. Luis Rodriguez says:

    Hola, he descargado tus archivos y estoy haciendo las pruebas respectivas, pero flash me da este error :

    SecurityError: Error #2010: Los archivos SWF locales del sistema de archivos no pueden utilizar sockets.
    at flash.net::Socket/internalConnect()
    at flash.net::Socket/connect()
    at KinectDemo_fla::MainTimeline/frame1()

    estoy usando flash player 10 y 10.1

  2. Luis Rodriguez says:

    y cuando le doy a continuar en lugar de descartar ese error, me sale el siguiente error :

    Error #2044: securityError no controlado: text=Error #2048: Violaci

  3. Luis Rodriguez says:

    Descubri que cuando ejecuto el swf directamente desde flash (osea con un control + enter ) todo funciona bien !!!

  4. Rhuno says:

    Judging from the errors, it looks like you were trying to run the swf on its own and running into security problems. It looks like you figured it out though (publishing from the IDE gets around it).

    You can also update your flash player settings to allow certain directories or files to access the internet.

  5. Luis Rodriguez says:

    Así es, gracias Rhuno por el tutorial, me ha servido para poder conectar mi kinect a flash y empezar a hacer algunas cosas con as3.

    Pero tengo una duda, por ejemplo quiero detectar la otra mano y la cabeza, es más, ampliar el área donde el esqueleto se mueve. Si pudieras indicarme el camino yo lo investigaría con gusto :)

  6. [...] empezar recomiendo mucho el Rhuno , explica detalladamente la conexión de kinect con flash, además de dejar las fuentes que nos [...]

  7. Luis Rodriguez says:

    He tratado de compilar el proyecto con microsoft visual estudio c++ 2010, pero me da un error, si pudieras indicarme con que programa fue hecho SkeletalTracking para quizás poder revisar el código y ver la manera de poder detectar la otra mano y la cabeza.

  8. Rhuno says:

    Hi Luis, the skeletal tracking program is written in C#, not C++ so you’ll need the visual C# compiler to build it. You can send along the left hand and head data by accessing the appropriate joints and adding them to byte array in C# to be sent over the socket. Something like this (see page 2 of the tutorial for reference):

    pos = sk.Joints[JointID.HandRight].ScaleTo(640, 480, .5f, .5f).Position;
    bw.Write(pos.X);
    bw.Write(pos.Y);

    pos = sk.Joints[JointID.HandLeft].ScaleTo(640, 480, .5f, .5f).Position;
    bw.Write(pos.X);
    bw.Write(pos.Y);

    bytes = ms.ToArray();
    sock.Send(bytes);

  9. Luis Rodriguez says:

    Muchas gracias por responder y tomarte la molestia de traducir mis preguntas para entenderlas en tu idioma.

    He logrado ya detectar las 2 manos, la cabeza y luego intentaré con las demás partes del cuerpo.

    Sobre lo del nodejs, tengo algunos amigos que tambien me dicen que no lo usarían y he visto que otros usan TUIO, te dejo el enlace por si en algún momento tienes tiempo de estudiarlo, yo lo revisaré en unos días y ya te cuento como me fué .

    http://bubblebird.at/tuioflash/guides/getting-started/

  10. Luis Rodriguez says:

    Hola, he estado haciendo ya algunas cosas con kinect y as3, te dejo algunos vídeos de mis avances con el fin de compartir experiencias:

    http://www.youtube.com/watch?v=s9ECUl5EKok&list=UUCkTDcrfKLmA

    http://www.youtube.com/watch?v=dvr1LYVQrr8&feature=player_detailpage

    He logrado tambien detectar todos los puntos del cuerpo, ya subiré un vídeo mas adelante, quisiera preguntarte ¿ tu tambien sientes ese tiempo lento mientras el kinect se comunica con as3, hay una especie de pequeño tiempo de espera “delay”?

  11. Jesus says:

    Alguien sabe como conectar flash con el TUIO???

  12. Luis Rodriguez says:

    Hola Jesús , pues sigue los pasos que aquí rhuno nos dejo y podrás hacerlo, yo pude, otra cosa hay un delay en la comunicación de flash y kinnect, pero imagino con el tiempo se solucionarán esos problemas

  13. danny says:

    I was curious if this was done with the Kinect SDk Beta 1?

    I cannot seem to get the SkeletalTracking project file to run without throwing this error:
    Could not load file or assembly ‘Microsoft.Research.Kinect, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified.

    Any ideas?

  14. Rhuno says:

    @danny

    Yes, this was done with the Beta 1 SDK. I haven’t tried it with the Beta 2 yet.

  15. Howdy. I simply want to congratulate you on obtaining another dedicated follower – ME!

  16. dany says:

    hello you can add sample for BETA 2? PLEASS
    THANKS
    BEST
    REGARDS

Leave a Reply

Subscribe to RSS feed FGS5 Badge