JMF based video capture (teaser post)
I've got an imminent deadline for a prototype of my 1st serious camera tracking project and I just can't seem get QT4Java to play nicely (make that "consistently") on my dev machine. Sometimes it works. More than often it does not...
Yesterday I wasted almost a full day with (un/re)installing various versions of QT and WinVDIG, reading and hacking - then decided to throw in the towel and try my luck with the Java Media Framework to get a camera feed into my app...
Another 12 hours later, after wading through Sun's JMF docs and various failed experiments, writing and merging little demos scavenged from across the web, I've got it working: Me waving back at myself in B&W on screen as JMF captured (and processable) video with Processing.
Disclaimers (and hence the "teaser" subtitle):
P2D (default renderer) 44fps average
P3D 52fps average
(*both fps counts over 1 minute) - I'm not sure if/how much this is faster than the default QT4Java solution...
Next steps are obviously to wrap this up in a library, but this probably won't be anytime until end of May since I want to do it properly. I might publish the existing code earlier, but it's currently using an Java interface mechanism for callbacks and so will only work outside the P5 IDE (e.g. in Eclipse)... watch this space!
Yesterday I wasted almost a full day with (un/re)installing various versions of QT and WinVDIG, reading and hacking - then decided to throw in the towel and try my luck with the Java Media Framework to get a camera feed into my app...
Another 12 hours later, after wading through Sun's JMF docs and various failed experiments, writing and merging little demos scavenged from across the web, I've got it working: Me waving back at myself in B&W on screen as JMF captured (and processable) video with Processing.
Disclaimers (and hence the "teaser" subtitle):
- So far only working under Eclipse
- JMF only exists for Windows and Linux (no OSX, sorry Robert!)
P2D (default renderer) 44fps average
P3D 52fps average
(*both fps counts over 1 minute) - I'm not sure if/how much this is faster than the default QT4Java solution...
Next steps are obviously to wrap this up in a library, but this probably won't be anytime until end of May since I want to do it properly. I might publish the existing code earlier, but it's currently using an Java interface mechanism for callbacks and so will only work outside the P5 IDE (e.g. in Eclipse)... watch this space!
9 Comments:
Yes, the JMF is quite hard to come by.. It would therefore be great to make it into a simple-to-use library for Processing.
I once interfaced Scott Klemmer's Papier-M�ch� with Processing. It makes good use of JMF and JAI (Java Advanced Imaging) for camera tracking purposes and performs very well.
(It's part of a larger tangible input framework, which makes it hard to extract, but could also be interesting as a whole, depending on your needs.)
BTW: There seems to be hope for Mac users. The all-Java version (no dlls) is reported to work on OS X, plus Sun is taking up development on JMF again - and is asking for requests..
Great work!
It's good to have a video alternative for Linux users.
Little mistake in your post. I think the default renderer of Processing is JAVA2D, I don't even think that P2D works at the moment.
Sorry for your pain man, but I look forward to test this alternative library :)
well i have the QTjava doing video and audio grabbing qt7 qtjava6.5 np...
there is an order to getting the gworld and u do have to make your own viewer... but once u get passed all that hassle u should be fine... drawing it to screen will be slow because you will have to do your own compatable image in java, and that tends to be piss poor (as u may have found)... if u are doing image tracking/diffrencing then u wont have a problem here...
the new way of doing this is to make an opengl shader and pass that into your code using jogl- maybe abit to dark if u are just starting out...
JMF is cool- but u will run into the hell of getting it to run anywhere in a stable manner... its also the slowest we found... shame SUN havent updated it in a looonnggg time (dont run on vista)...
well good luck with your developmentz - yell if u need help dude... always down the A&P... didnt no u have a blog ;)
Thanks for all the feedback guys!
Andr�, thanks for the links and very interesting to hear about Sun's re-animation efforts for JMF. Have you got any links to back this up (I couldn't find anything recent)...?
Anonymous - do you happen to be known as Ant? ;) Thanks for the info. Like the idea with the JOGL shader, will have to enquire more. I eventually got QT4Java to work too (although my problems were different than described in the P5 FAQ), but in my tests JMF is running visibly faster and more stable than using the QT based Processing video library...
I found this message on the JMF-Interest mailing list from December '05, with plenty of replies in the following months. So there might be hope - though there's also scepticism (Be sure to also read the comments). No matter what the outcome will be, I think JMF would be a better way to go for Processing than QT.
Also just found a how-to for installing JMF with OS X.
Please, for the love of all that is holy, make this work on OSX!!! I want what I dont have!!!
indeed toxi :) aNt here.... i have had a chat with some of the dudes at Apple on speeding up QTJ when it comes to capture (older versions where faster)- then QT was updated to use Cocoa, there was alot of the API that was handed over to Quarts Extream (Core Video wasnt ready). The offshot of that was QTJ was cut up because of the way Java was rendering back then (super god Bill left QTJ and started at Core Audio didnt help). So capture was still there, but its a hack to get it workin (buffered to render isnt good).
QTJ for QT7 should still be grabbing your pixel data faster then JMF mind. its native to modern cards. i'm hanging on for Lepeard and Core Video atm, the Vista video stuff looks like a nightmare mind.
our onion has done a super real fast wrapper for video grabbing (windows) wonder if he will splash is public.. i kick him... was very nice indeed for the tracking stuff we did...
Do you find video playback is slow in from a QTJCanvas? this is a good test to see if your Java drawing is upto it. Also setting the native pixel format for the capture speeds stuff up alot. If we like i can look into getting the Java going faster also (on macosx anyways).
FMJ (open-source implementation of JMF) also has video capture device support, via the lti-civil project, which is lower-level. Supports Mac OS X, Windows, Linux, 64-bit Linux.
http://fmj.sourceforge.net
http://sourceforge.net/projects/lti-civil/
Ken Larson
FMJ project.
Post a Comment
Links to this post:
Create a Link
<< Home