The OV538 Based PS3Eye Camera (Version B4.04.27.1)

This newer version of the PS3Eye camera features less parts. In this version Sony removed the GL850A USB2.0 hub thus greatly simplifying their design. Check here if you want to see the internals of the older version of this camera.

 
Here you can see the OV538 USB 2.0 camera controller chip.

 
I marked the I2C signals that can be used to dump the commands sent to the OV7720 as well as data being read from the EEPROM during camera’s boot sequence.

 
The front side of camera’s PCB contains minimal amount of parts in comparison the older camera version.

 
Both versions of PS3Eye camera feature 24C64 (8K) EEPROM chip that contains USB descriptor table as well as the camera’s firmware. Check here for camera firmware difference analisys. I checked the status of the WP pin and it is tied to the GND (EEPROM is not write protected!). This means that camera’s firmware is software upgradeable. By disconnecting (floating) pin 8, you’ll get the OmniVision’s default (OV0534) PID and VIDs (Not recomended unless you are a firmware developer or want to modify camera’s firmware)

Sony PS3EYE (B3.04.06.1) IR Filter Removal and More…

After playing with the software part and getting PS3Eye to capture under Windows, I decided to see how difficult is to remove the IR blocking filter from the lens of this camera. Please note that this tutorial is only for experienced users who are interested in using the PS3Eye camera for multitouch applications.

NOTE: The pictures used in this tutorial refer to the camera version B3.04.06.1. This is the older version of PS3Eye camera and features the OV534 USB controller chip. For the pictures of the newer version (B4.04.27.1) of PS3Eye camera featuring OV538 chip scroll down. 
BTW, this tutorial applies to both versions of PS3Eye camera.

I recorded the whole process in a few pictures. In addition I took a closer look at this camera’s chipset.

 
First remove the four screws at the back of the camera.

 
After carefully removing the back cover, you will see the camera’s main board.

 
To remove the front cover of the camera, remove another four screws.

 
Now you’ll have a full access to the camera’s PCB. Note that four microphones are further protected by a plastic cover.

Lets take a closer look at the chipset.

 
The main camera USB2.0 controller chip by OmniVision.

 
The GL850A USB2.0 hub controller chip.

 
Here you see the two clock crystals. One for each chip.

 
The 8K I2C EEPROM chip. This is where the Sony’s camera/audio vendor IDs, product IDs and firmware are stored. By disconnecting (floating) pin 8, you’ll get the OmniVision’s default (OV0534) PID and VID (Not recomended unless you are a firmware developer or want to modify camera’s firmware)

 
This is where the Sony’s GL850A hub vendor and product IDs are stored. By disconnecting (floating) pin 1, you’ll get the Genesys Logic’s default (GL850A) IDs. (Again, not recomended unless you are a firmware developer or want to modify camera’s firmware)

Now, lets remove the lens and IR blocking filter.

 
To do this remove the two CMOS lens screws.

 
Carefully pull off the lens assembly to reveal the OV07720 CMOS sensor.

 
The IR blocking lens is mounted on the inside of the lens, close to the CMOS sensor. It is heat-pressed in, so it is fairly easily removed.

 
After cutting around the IR filter with a sharp knife, you should be able to just pop it out.

 
Here you see the IR blocking filter plastic socket.

 
I was prying to much on the IR filter and cracked it.

Now just follow the reverse steps to put the camera together.

Your camera shoud be ready for Multitouch/IR vision applications. Please note that camera sensor is very sensitive to the IR light which makes it even more attractive (besides the high capture frame rates) for these applications.

Sony PS3Eye Camera on Windows OS

Well boys and girls, I’ve been working hard last few weeks to make this great camera work under Windows.

As you may seen it before, here are the specs:

- 4 channel audio input:16 bits/channel, 48kHz, SNR 90db 
- 56º or 75º Field of View zoom lens 
- 2.1 F-stop, <1% distortion, fixed focus (25cm to 8 at 75º FOV) 
- 640 x 480 at 60 frames/second 
- 320 x 240 at 120 frames/second 
- USB.0 high-speed data transfer 
- Uncompressed video or optional JPEG compression

This makes the PS3Eye ideal for multitouch applications. The best part is the price $39.99! I found mine here.

Now, the main problem with this camera is that there are no drivers for Windows. The camera’s chipset info is virtually non-existent on the web. 
After examining the camera internals (pictures here) I found that it features the OV534-LB50 camera USB 2.0 bridge and the OV7720 CMOS VGA sensor. Both of these are made by OmniVision.

I started thinking to my self: “This camera is awesome and it will be such a great and inexpensive replacement for Firefly MV and the like. If we could just get it to work under Windows…”

Initially, I started poking around with the USB trying to send some commands to the PS3Eye and see what happens…

After many long nights I’m bringing you the result: 
- Full VGA (640×480) 60fps video capture test app that features uncompressed high quality raw video 
- Low CPU overhead (since there is no decompression involved on the PC) 
- Very low latency (1 frame time period)

The camera currently streams video in YUYV format, therefore each frame is 640*480*2 bytes. 
At 30fps this amounts to about 17.5MB/s which is pretty low in comparison to the total USB 2.0 bandwidth. 
At 60fps the amount of data gets higher and it could be affected by other peripherals connected to the USB host controller. 
This is why it is recommended that the camera be the only device connected to the USB host controller.

Most of the CPU overhead that I currently have is the color conversion code that is implemented in straight C/C++ without any SIMD optimizations. 
For real (MT) applications this code will go away, since we will be extracting raw grayscale image (every second byte of YUYV).

My driver exposes PS3Eye camera as a device with direct access, thus eliminating the complexities and the overhead of DirectShow system. 
For multitouch applications (where low latency is a key) I will be working on custom PS3EYE capture filter for use in TouchLib. In parallel I will be working on a DirectShow filter that will allow wide use of this camera on Windows. 
NOTE: I am currently running Vista and all the code is developed and tested under this particular OS, but it should work on XP with no problems.

Installation:

- Download and run PS3EyeSetup file.
- Click ‘Install’ and follow the setup process.
- Plug in the camera.

After successful installation, your Device Manager should look similar to this:

Now run the PS3EyeTest.exe program, and the captured video as well as the FPS counter will be displayed.

Go, try it for yourself…

Enjoy!

I created a new single PS3EyeSetup that will autmoatically install all the necessary files.
For more info and up-to-date files go here.

*UPDATE* 10-16-2008
New driver allows the camera to run under x86 and x64 Windows systems.
New TouchLib filter allows the use of this great camera in multitouch applications!!!
Incredible capture frame rates and image quality!!!
Read more here…

If you find the software useful or if you feel like supporting this project, please feel free to click the donation link below.

Dual IR Laser, Firefly MV, TouchLib running @ 112fps + TouchLib Speed/CPU Fix !!!

Today I got the Firefly MV B/W camera from PGR. I also added second IR laser to my original setup. This eliminates finger occlusions that were present in my previous single laser setup. 
I took some videos of this setup running demo apps, so I thought I showcase it here.

The TouchLib was modified (for fast blob processing) so that now runs synchronously to the camera capture rate. It also outputs current blob processing frame rate.

Here is the OSC app:

Here is the Fire demo:

Here is the Photo app:

Here is the Wall Pong app:

Here is the Puzzle app:

Here is the Ripples app:

Here is the Smoke app:

*** UPDATED ***

Dual IR laser blobs (camera view):

Calibration process (here I use ROI feature of the Firefly MV camera @ 320×240 frame size):

Many of you asked about my version of TouchLib with speed/CPU fix shown above. You can get it below and test it for yourself. Please note that I am using DSVideoLib as the input capture filter. The library was tested and it functions properly with this filter only. With other filters (such as VideoWrapper) the CPU usage may be very high, so please use it at your own risk.

Enjoy!

~Alex

Files:
TouchLib.part1.rar
TouchLib.part2.rar
TouchLib.part3.rar

LLP Multitouch Zero Force Setup Take #2

Every since posting the results of my previous setup, I worked on improving some aspects of this setup. 
During this time I decided to make a vertical version and at the same time test my new 850nm 10mW laser that this time featured more advanced version of the line generating lens. This reflected in fully uniform brightness across the line width and wider coverage angle. 
Because of all of this, I found that using only one laser was sufficient enough for simple multitouch operation.

See it for yourself…

Here are the results:

 
 
 
 
 
 
 
 
 
 
 
 
 

In action:

Sun light front view:

Sun light rear view:

Enjoy!

Multitouch Zero Force Setup Based on IR Laser Light

Here is the setup that I’ve been working on for a while now. It is based on front IR illumination by IR laser.

Camera

For this project I’ve been using my old Orange Micro iBot firewire camera with great success. This low-cost camera generates 640×480 at 30fps uncompressed images in Y800 format. I found that because of all of this and the IR sensitivity this camera is very suitable for IR multitouch application:

After removing the original lens and mounting the new lens without IR blocking filter, I placed the double-sided tape on top of the lens and cut the opening in it:

IR Bandpass Filter

In the past I used photo negative with some success, but found that using narrow bandpass filter is by far the best in removing any unwanted IR noise and visible light:

Here is the filter mounted on the camera:

Table Setup

Currently I’m in the process of redesigning the table. Right now it is an open-frame setup:

IR Laser

And now the most important part. I use the 850nm IR Laser light to illuminate the front of the surface. When finger or any object interfere with the laser light, the camera below picks up the bright spot and runs it through the image processing algo to generate detected blob information. 
Here is the view from the surface:

The laser is mounted on a tripod on a tri-point adjustable platform so that it can be positioned is a way that it generates a light beam that is parallel to the glass surface:

The IR laser is positioned just above the glass surface:

Software

Even though I could use touchlib for blob processing/tracking, having a lot of experience in image processing, I found that it was easier for me to write a highly optimized version blob tracking software. Because of this, I achieve very low latency and low CPU usage running my blob tracking software (around 3-4% on my E6600 2.4GHz system). 
Here are some screenshots of my blob tracking app: 

As you can see blobs are very bright and clean:

Here is my iPhone placed on the surface:

Video

Here you can see the system in action. While using FTIR the blobs become dimer during the fast movement, using the laser method the blobs are always bright

Short video of the system: