Sony PS3Eye Camera TouchLib, DirectShow and more…

•October 17, 2008 • 52 Comments

I’m pleased to announce the new release of the PS3Eye driver. The camera code had been completely rewritten from the ground up. The driver now uses Microsoft’s WinUSB driver. I decided to go this route since the old driver had some performance issues. It is strange that Sony have chosen to stream video over USB bulk transfer pipe. Usually, video and audio USB devices use isohronous transfers since they have higher priority. On a slower systems this results in the video frame data drop resulting in shifted (out of sync) image.

Anyways, the good new is that this time around the driver will work with both x86 and x64 version of Windows (XP and Vista). This will hopefully make many of you who were asking for x64 version very happy.

Installation

Please make sure you completely remove any previous versions of PS3Eye drivers and software. 
You can easily check if this is done properly by plugging the camera checking if Windows detects and automatically installs the driver. 
If it doesn’t, the driver has been removed. To install download and run the latest PS3EyeSetup file and follow the instructions. 
If your camera is not automatically detected when you plug it in, point the device manager to the Driver directory under the installation path. The driver will install and your camera is now ready. 
VC++ 2005 SP1 redistributable is also included in the PS3EyeSetup for your convenience.

PS3Eye as DirectShow Capture Device

The PS3Eye.ax is the DirectShow video capture component that is automatically registered by the PS3EyeSetup. It exposes the camera frames in the RGB24 color format for better software compatibility. Here is the list of currently supported resolutions and frame rates:

  • 320×240 @ 15fps 
  • 320×240 @ 30fps 
  • 320×240 @ 60fps 
  • 320×240 @ 75fps 
  • 320×240 @ 100fps 
  • 320×240 @ 125fps 
  • 320×240 @ 150fps (experimental) 
  • 640×480 @ 30fps 
  • 640×480 @ 40fps 
  • 640×480 @ 50fps 
  • 640×480 @ 60fps 
  • 640×480 @ 75fps (experimental)

PS3Eye TouchLib Build

I am very excited to announce the first TouchLib build that now fully supports the PS3Eye camera. All of the above resolutions and frame rates are supported in the current build. 
I implemented a new TouchLib filter called ps3eyecapture that loads the PS3Eye driver directly and allows the use of this device in multitouch applications. The direct to PS3Eye camera through the driver guaranties the minimal capture latency.

To select the desired resolution and frame rate, just uncomment the format parameter line as shown here:

TouchLibFilterGraph.jpg

NOTE: Please make sure you backup the config.xml file before running TouchLib since it will overwrite it on exit and you will lose all of the commented formats.

And of course, the files: PS3EyeSetup (v2.0b81109) and TouchLib (PS3Eye) build .

Enjoy!

If you find the software useful or if you feel like supporting this project, please feel free to click the donation link below.

Advertisements

Sony PS3Eye Camera DirectShow Capture Source Filter *Update* v3.0.0.0901

•October 2, 2008 • 955 Comments

Well, here it is… 
I managed to put together a first version of the DirectShow capture filter for PS3Eye camera. 
This will allow us to use this great camera in various Windows applications (assuming they are compatible with DirectShow system) including TouchLib for use in multitouch applications.

I made single PS3EyeSetup application that will install PS3Eye driver, DirectShow filter and test applications all at once. It will also allow for clean Unistall, for those of you that are experiencing problems with Flash content in IExplorer and Mozilla browsers. Please note that before running Uninstaller you must close any application that uses PS2Eye camera or some of the camera component will not be properly removed.

Installation Steps:

– Unplug the PS3Eye camera from your comupter.
– Uninstall any previous version of PS3Eye software.
– Download and run the latest PS3EyeSetup file.
– Click ‘Install’ and follow the setup process.
– Plug in the camera.

After successful installation, your Device Manager should look similar to this: 

You can run the PS3EyeTest application from the start menu at this point to verify if camera installed properly. You’ll see something like this:

I added the option of controlling the camera settings such as Gain, Exposure, White balance etc.

DirectShow Video Source for PS3Eye camera will also be installed, registered and ready for use. To test if PS3Eye DirectShow component functions properly, run the AmCap application from the start menu.

After a few moments the camera will start capturing and the AMCAP will show the camera preview in its window: 

Camera properties selects video format:

Camera settings:

Currently the PS3Eye camera DirectShow component is set by default to capture uncompressed 640×480 RGB (24bit) video at 30fps. 

Now you can enjoy using this great camera in your favorite Windows app!

NOTE: Thank you all for your feedback. For those of you who are experiencing problems, you can now uninstall the software by running Uninstall from the start menu. If for some reason the camera DirectShow files (PS3Eye.ax and PS3EyeLib.dll) are not removed and still on your system, follow these steps to manually remove them:

Manual removal of PS3Eye.ax and PS3EyeLib.dll files:

– Close any program that are using PS3Eye camera.
– Press ‘WinKey’+R on your keyboard or alternatively go to Start->Run
– Type the following: regsvr32 /u “C:\Program Files\AlexP\PS3Eye.ax”
– You will see the following dialog box confirming successful un-registration of the PS3Eye.ax file.

– Now you can safely delete PS3Eye.ax and PS3EyeLib.dll from your machine.

If the manual removal steps above fail for some reason, run your Windows in safe mode and repeat these steps.

To Do List:

– Make Setup application
– Create camera property pages
– Add camera resolution selection (640×480, 320×240, 160×120)
– Add selectable camera capture window (x, y, width, height)
– Add capture frame rate adjustment (640×480@60fps max, 320×240@120fps max)
– Add selection of Auto/Manual camera operation modes
– Manual adjustment for Exposure and Gain
– Add RGB-16/24/32 color output format (this will increase compatibility with the software out there)
– Implement 64-bit version of the PS3Eye driver

I am currently working on implementing these features. As am only working part-time on this project please be patient and check here for future updates.

NOTE: Since currently I do not control the PS3Eye’s microphone (detected and installed by Windows by default), there is no way for me to make sure that every program that uses both camera and microphone will work properly. I am aware that there are some issues with this when running WML and other programs on some machines. I am currently looking into this and possibly a way to find a workaround.

And most importantly, the PS3Eye Setup file:

NOTE: The older setup files are crossed but still available below. For the latest setup scroll to the bottom.

PS3EyeSetup (v1.0b81006)

Notes (v1.0b81006): I worked on major bug fix. Flash player camera capture now works properly. I successfully tested it on http://www.cameroid.com

PS3EyeSetup (v1.0b81007)

Notes (v1.0b81007): Increased compatibility by setting default capture frame rate to 15fps for DirectShow filter (320×240), making it fully compatible with Flash and many video conferencing programs out there. Fixed ‘green screen’ bug.

PS3EyeSetup (v1.0b81008)

Notes (v1.0b81008): Fixed preview/capture issue with AMCAP and other programs. The camera resources are properly released now.

PS3EyeSetup (v2.0b81019)

Notes (v2.0b81019): Created DirectShow camera property page (selectable resolution and frame rate). Support for RGB-16/24/32 color output format. Implemented both 32-bit and 64-bit version of the PS3Eye driver. Fixed the PS3EyeCamera.inf file so that drivers install correctly. Included PS3EyeLib SDK as described here.

PS3EyeSetup (v2.0b81021)

Notes (v2.0b81021): Driver installation tested on both Vista x86 and x64 it now works properly and driver files are found correctly. As of this version (v2.0b81021) the PS3EyeLib driver uses the advanced thread scheduling features only available in Vista. This greatly improves its performance and stability on this OS.

PS3EyeSetup (v2.0b81029)

Notes (v2.0b81029): New version of driver now supports resolution selection as well as the adjustment of the camera settings such as Gain, Exposure, White balance etc. Please make sure you fully uninstall any previous version of PS3Eye software you have on your machine before installing this latest release.

PS3EyeSetup (v2.0b81109)

Notes (v2.0b81109): Fixed crash problems with various browsers involving Flash content, including crashes when going into ‘Settings’ option in the embedded Flash player. This also fixes problems some experienced with MSN, Skype and others. Succesfully tested the new version on the Cameroid website. Please make sure you fully uninstall any previous version of PS3Eye software you have on your machine before installing this latest release.

PS3EyeSetup (v2.0b81111) (MD-5: 7DCDCDBF6B0E184AEAA8E709F259E576)

Notes (v2.0b81111): Fixed bug that caused errors in AMCAP in the previous version. At the same time fixed error issues in Flash player. Some of you reported that there is a worm in a setup file. After investigating the issue I found out that my ISP’s server was infected. They fixed the issue and I uploaded the setup file again. I included the MD-5 checksum of the original file. I tested this file with both Kaspersky and Avast! and it is clean.

The camera was tested and works with Skype. Here are some screenshots of camera setup window in Skype.

Skype video:

Camera settings:

PS3EyeSetup (v2.0b81231) (MD-5: EB14A19766BE5DEC8340E78C6AC03819)

Notes (v2.0b81231): Fixed issue with ‘file in use’ (write protected) PS3Eye.ax file. This was due to running installer/uninstaller while MSN messenger is opened. Fixed annoying ‘software upgrade’ popup.

PS3EyeSetup (v2.1.0.0130) (MD- Continue reading ‘Sony PS3Eye Camera DirectShow Capture Source Filter *Update* v3.0.0.0901’

Differences Between B3.04.06.1 and B4.04.27.1 PS3Eye Camera Firmware

•September 15, 2008 • 5 Comments

The analysis of the differences between B3.04.06.1 and B4.04.27.1 firmware revealed that the only difference between them is the reported amount of power that the device consumes. This information is stored in configuration descriptor in the camera’s EEPROM. 
The B3.04.06.1 version reports 10mA and B4.04.27.1 reports 500mA. This change is due to the fact that in the newer version of the camera the OV538 chip sits directly on the USB bus as opposed to OV534 chip being behind the GL850A USB2.0 hub controller chip.

PS3Eye firmware B3.04.06.1 dump:

PS3Eye firmware B4.04.27.1 dump:

The first two differences belong to the camera device descriptor and refer to the current consumption of this device.

The OV538 Based PS3Eye Camera (Version B4.04.27.1)

•September 10, 2008 • 10 Comments

This newer version of the PS3Eye camera features less parts. In this version Sony removed the GL850A USB2.0 hub thus greatly simplifying their design. Check here if you want to see the internals of the older version of this camera.

 
Here you can see the OV538 USB 2.0 camera controller chip.

 
I marked the I2C signals that can be used to dump the commands sent to the OV7720 as well as data being read from the EEPROM during camera’s boot sequence.

 
The front side of camera’s PCB contains minimal amount of parts in comparison the older camera version.

 
Both versions of PS3Eye camera feature 24C64 (8K) EEPROM chip that contains USB descriptor table as well as the camera’s firmware. Check here for camera firmware difference analisys. I checked the status of the WP pin and it is tied to the GND (EEPROM is not write protected!). This means that camera’s firmware is software upgradeable. By disconnecting (floating) pin 8, you’ll get the OmniVision’s default (OV0534) PID and VIDs (Not recomended unless you are a firmware developer or want to modify camera’s firmware)

Sony PS3EYE (B3.04.06.1) IR Filter Removal and More…

•September 5, 2008 • 12 Comments

After playing with the software part and getting PS3Eye to capture under Windows, I decided to see how difficult is to remove the IR blocking filter from the lens of this camera. Please note that this tutorial is only for experienced users who are interested in using the PS3Eye camera for multitouch applications.

NOTE: The pictures used in this tutorial refer to the camera version B3.04.06.1. This is the older version of PS3Eye camera and features the OV534 USB controller chip. For the pictures of the newer version (B4.04.27.1) of PS3Eye camera featuring OV538 chip scroll down. 
BTW, this tutorial applies to both versions of PS3Eye camera.

I recorded the whole process in a few pictures. In addition I took a closer look at this camera’s chipset.

 
First remove the four screws at the back of the camera.

 
After carefully removing the back cover, you will see the camera’s main board.

 
To remove the front cover of the camera, remove another four screws.

 
Now you’ll have a full access to the camera’s PCB. Note that four microphones are further protected by a plastic cover.

Lets take a closer look at the chipset.

 
The main camera USB2.0 controller chip by OmniVision.

 
The GL850A USB2.0 hub controller chip.

 
Here you see the two clock crystals. One for each chip.

 
The 8K I2C EEPROM chip. This is where the Sony’s camera/audio vendor IDs, product IDs and firmware are stored. By disconnecting (floating) pin 8, you’ll get the OmniVision’s default (OV0534) PID and VID (Not recomended unless you are a firmware developer or want to modify camera’s firmware)

 
This is where the Sony’s GL850A hub vendor and product IDs are stored. By disconnecting (floating) pin 1, you’ll get the Genesys Logic’s default (GL850A) IDs. (Again, not recomended unless you are a firmware developer or want to modify camera’s firmware)

Now, lets remove the lens and IR blocking filter.

 
To do this remove the two CMOS lens screws.

 
Carefully pull off the lens assembly to reveal the OV07720 CMOS sensor.

 
The IR blocking lens is mounted on the inside of the lens, close to the CMOS sensor. It is heat-pressed in, so it is fairly easily removed.

 
After cutting around the IR filter with a sharp knife, you should be able to just pop it out.

 
Here you see the IR blocking filter plastic socket.

 
I was prying to much on the IR filter and cracked it.

Now just follow the reverse steps to put the camera together.

Your camera shoud be ready for Multitouch/IR vision applications. Please note that camera sensor is very sensitive to the IR light which makes it even more attractive (besides the high capture frame rates) for these applications.

Sony PS3Eye Camera on Windows OS

•September 3, 2008 • 50 Comments

Well boys and girls, I’ve been working hard last few weeks to make this great camera work under Windows.

As you may seen it before, here are the specs:

– 4 channel audio input:16 bits/channel, 48kHz, SNR 90db 
– 56º or 75º Field of View zoom lens 
– 2.1 F-stop, <1% distortion, fixed focus (25cm to 8 at 75º FOV) 
– 640 x 480 at 60 frames/second 
– 320 x 240 at 120 frames/second 
– USB.0 high-speed data transfer 
– Uncompressed video or optional JPEG compression

This makes the PS3Eye ideal for multitouch applications. The best part is the price $39.99! I found mine here.

Now, the main problem with this camera is that there are no drivers for Windows. The camera’s chipset info is virtually non-existent on the web. 
After examining the camera internals (pictures here) I found that it features the OV534-LB50 camera USB 2.0 bridge and the OV7720 CMOS VGA sensor. Both of these are made by OmniVision.

I started thinking to my self: “This camera is awesome and it will be such a great and inexpensive replacement for Firefly MV and the like. If we could just get it to work under Windows…”

Initially, I started poking around with the USB trying to send some commands to the PS3Eye and see what happens…

After many long nights I’m bringing you the result: 
– Full VGA (640×480) 60fps video capture test app that features uncompressed high quality raw video 
– Low CPU overhead (since there is no decompression involved on the PC) 
– Very low latency (1 frame time period)

The camera currently streams video in YUYV format, therefore each frame is 640*480*2 bytes. 
At 30fps this amounts to about 17.5MB/s which is pretty low in comparison to the total USB 2.0 bandwidth. 
At 60fps the amount of data gets higher and it could be affected by other peripherals connected to the USB host controller. 
This is why it is recommended that the camera be the only device connected to the USB host controller.

Most of the CPU overhead that I currently have is the color conversion code that is implemented in straight C/C++ without any SIMD optimizations. 
For real (MT) applications this code will go away, since we will be extracting raw grayscale image (every second byte of YUYV).

My driver exposes PS3Eye camera as a device with direct access, thus eliminating the complexities and the overhead of DirectShow system. 
For multitouch applications (where low latency is a key) I will be working on custom PS3EYE capture filter for use in TouchLib. In parallel I will be working on a DirectShow filter that will allow wide use of this camera on Windows. 
NOTE: I am currently running Vista and all the code is developed and tested under this particular OS, but it should work on XP with no problems.

Installation:

– Download and run PS3EyeSetup file.
– Click ‘Install’ and follow the setup process.
– Plug in the camera.

After successful installation, your Device Manager should look similar to this:

Now run the PS3EyeTest.exe program, and the captured video as well as the FPS counter will be displayed.

Go, try it for yourself…

Enjoy!

I created a new single PS3EyeSetup that will autmoatically install all the necessary files.
For more info and up-to-date files go here.

*UPDATE* 10-16-2008
New driver allows the camera to run under x86 and x64 Windows systems.
New TouchLib filter allows the use of this great camera in multitouch applications!!!
Incredible capture frame rates and image quality!!!
Read more here…

If you find the software useful or if you feel like supporting this project, please feel free to click the donation link below.

Dual IR Laser, Firefly MV, TouchLib running @ 112fps + TouchLib Speed/CPU Fix !!!

•June 14, 2008 • 3 Comments

Today I got the Firefly MV B/W camera from PGR. I also added second IR laser to my original setup. This eliminates finger occlusions that were present in my previous single laser setup. 
I took some videos of this setup running demo apps, so I thought I showcase it here.

The TouchLib was modified (for fast blob processing) so that now runs synchronously to the camera capture rate. It also outputs current blob processing frame rate.

Here is the OSC app:

Here is the Fire demo:

Here is the Photo app:

Here is the Wall Pong app:

Here is the Puzzle app:

Here is the Ripples app:

Here is the Smoke app:

*** UPDATED ***

Dual IR laser blobs (camera view):

Calibration process (here I use ROI feature of the Firefly MV camera @ 320×240 frame size):

Many of you asked about my version of TouchLib with speed/CPU fix shown above. You can get it below and test it for yourself. Please note that I am using DSVideoLib as the input capture filter. The library was tested and it functions properly with this filter only. With other filters (such as VideoWrapper) the CPU usage may be very high, so please use it at your own risk.

Enjoy!

~Alex

Files:
TouchLib.part1.rar
TouchLib.part2.rar
TouchLib.part3.rar