Falcon Christmas

The Lab (Technical Talk) => New Software Projects => Topic started by: djohnson on November 27, 2013, 05:11:39 PM

Title: an online sequencer
Post by: djohnson on November 27, 2013, 05:11:39 PM
I am working on an online based sequencer (think Single Page Application) that allows for the display design and layered effects sequencing.
output will generate fseq and seq files natively (initially)
any interested contributors?
Title: Re: an online sequencer
Post by: twooly on November 27, 2013, 05:29:22 PM
What language?
Title: Re: an online sequencer
Post by: djohnson on November 27, 2013, 07:57:36 PM
web api back end (C#) front-end angular js for framework, fabric.js for graphics, breeze js for client dal
Title: Re: an online sequencer
Post by: David Pitts on November 30, 2013, 11:51:56 AM
Quote from: djohnson on November 27, 2013, 05:11:39 PM
I am working on an online based sequencer (think Single Page Application) that allows for the display design and layered effects sequencing.
output will generate fseq and seq files natively (initially)
any interested contributors?

I am definitely interested. I Googled the js libraries, seems like some cool stuff.
Title: Re: an online sequencer
Post by: Frankr on December 03, 2013, 10:32:00 PM
I agree with David Pitts. After looking at these tool kits I am intrigued to see what you have in mind.

Frank
Title: Re: an online sequencer
Post by: djohnson on December 07, 2013, 03:32:13 PM
maybe we could setup a call to chat about the work I have in place, UX concepts and breaking up the work.
Title: Re: an online sequencer
Post by: sielbear on December 22, 2013, 09:45:43 AM
This gets pretty interesting if you think about adding video on the backend for "auto sequencing".  Essentially load your display elements online, define channel outputs, etc., then build your own video or use a pre-canned solution.  I'm sure there are audio use rights issues, however, if iTunes can sell a song for $1, I bet there's a deal to be made that would bundle the song with the download, etc.  Food for thought...
Title: Re: an online sequencer
Post by: djohnson on December 24, 2013, 12:39:41 AM
Quote from: sielbear on December 22, 2013, 09:45:43 AM
This gets pretty interesting if you think about adding video on the backend for "auto sequencing".  Essentially load your display elements online, define channel outputs, etc., then build your own video or use a pre-canned solution.  I'm sure there are audio use rights issues, however, if iTunes can sell a song for $1, I bet there's a deal to be made that would bundle the song with the download, etc.  Food for thought...
exactly where I was going... ;)
Title: Re: an online sequencer
Post by: djohnson on January 27, 2014, 02:49:42 PM
making excellent progress, audio licensing will be integrated in the solution.
more info, screenshots and... will be shared by end of February.
Title: Re: an online sequencer
Post by: tjames on December 09, 2014, 07:44:07 PM
Did anything ever come of this?
Title: Re: an online sequencer
Post by: Frankr on December 10, 2014, 11:41:36 AM
Haven't heard anything from David in a while and was wondering whatever came of this myself recently.  David is clearly one of the leaders in thinking about sequencing. LSP is still conceptually years ahead of everyone else and he left that about 2 years ago if memory serves.

Frank
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 12:28:50 PM
Quote from: Frankr on December 10, 2014, 11:41:36 AM
Haven't heard anything from David in a while and was wondering whatever came of this myself recently.  David is clearly one of the leaders in thinking about sequencing. LSP is still conceptually years ahead of everyone else and he left that about 2 years ago if memory serves.

Frank

I ask because last year I started building a sequencer targeted towards OSX. I own a dev firm and we primarily do web and mac app development. So while I have Windows machines, they aren't very good so using LSP just isn't in the cards since it requires a fairly powerful pc, which I'm for the most part unwilling to purchase solely to blink some lights :). Some of the concepts that I worked out focus primarily on usability since the learning curve for most of these programs is fairly steep, especially going from one to the other.

I had a couple prototypes built for different features I wanted to implement (like controlling the display via kinect and leap motion. Also toyed with visual sequencing, i.e. essentially painting on effects with the mouse via webgl), as well as a bare bones sequencer, and they were all built in the browser (technically in a webkit webview using my MacGap platform https://github.com/MacGapProject/MacGap2). With a little effort I could port them out to just run solely in the browser, or even port them over to a different platform like atom-shell or node webkit and they could control lights directly as well (I wrote a E1.31 client for Node.js that could be used in either of those platforms since they both integrate node)

Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 12:49:24 PM
Just get rid of the grid sequencing and I may have a chance!!  Lots of folks started something but never really saw the light of day. Someone 2 years ago was working on an element based sequencing, where you could simply say I want that tree to do this and that eve to do that etc. No working in tiny lil grids..

I would pay money if tools were developed that would allow a non artistic person like me to create a neat synchronized display (oh I can do it, have done it but it takes me a long time and it's really not fun at this point).

Tory
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 01:00:42 PM
Quote from: Livermore-Dad on December 10, 2014, 12:49:24 PM
Just get rid of the grid sequencing and I may have a chance!!  Lots of folks started something but never really saw the light of day. Someone 2 years ago was working on an element based sequencing, where you could simply say I want that tree to do this and that eve to do that etc. No working in tiny lil grids..
Tory

That's basically what I had intended with the 'visual sequencing' (didn't really have a better name for it). You'd basically sequence your show in the visualizer instead of a grid. You'd set up the visualizer like you do now with most programs (add a picture, draw/add your elements, etc), and then when you're ready to start sequencing you simply draw/overlay your colors/effects on your elements. If you have a time frame selected (or portion of the song), it would run that effect on those elements during that time frame, otherwise it just turns the element on/runs the effect to the end or until you overwrite the effect with another.

Title: Re: an online sequencer
Post by: Frankr on December 10, 2014, 01:30:16 PM
Most of my display is done using video files that are rendered against a whole house model in Nutcracker. my Let it go sequence took me 3 hours in sony movie studio cropping the video the way I wanted and boom done.  MUCH easier... Plus it is very easy to share the video files.

Nutcraker is the closest thing to element based sequencing. I can usually bust out a new sequence in 4-5 hours vs. the 30 it used to take me in LOR or LSP.

Still room for the technologies and techniques to grow though...

Frank
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 01:35:01 PM
Quote from: Frankr on December 10, 2014, 01:30:16 PM
Most of my display is done using video files that are rendered against a whole house model in Nutcracker. my Let it go sequence took me 3 hours in sony movie studio cropping the video the way I wanted and boom done.  MUCH easier... Plus it is very easy to share the video files.

I've never heard of this technique or how to handle this in xlights/nutcracker, is there some video functionality I'm not aware of? What exactly are you doing with the video?
Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 01:42:30 PM
Quote from: Frankr on December 10, 2014, 01:30:16 PM
Most of my display is done using video files that are rendered against a whole house model in Nutcracker. my Let it go sequence took me 3 hours in sony movie studio cropping the video the way I wanted and boom done.  MUCH easier... Plus it is very easy to share the video files.

Nutcraker is the closest thing to element based sequencing. I can usually bust out a new sequence in 4-5 hours vs. the 30 it used to take me in LOR or LSP.

Still room for the technologies and techniques to grow though...

Frank

Okay Frank, I've grabbed a glass of water and have pulled up a chair!!

???? Que?!

Tory
Title: Re: an online sequencer
Post by: arw01 on December 10, 2014, 03:23:33 PM
Brings a different flavor, that to me becomes light animation to music vs very precise placement in the grid.  Room for both and with David's video macro he released for LSP you can do both very seemlessly in LSP.

I suppose I should figure out how to do the timing marks in audacity with it's tools, would shave a few hours off my LSP time.

David hasn't posted anywhere I have been for a few months.  LSP is way along in the re-write and it promisses to be a totally different animal when done.

There was a fellow working on an independent visualizer, I would like to see a 3d one of those  ;P
Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 05:26:27 PM
Quote from: arw01 on December 10, 2014, 03:23:33 PM
Brings a different flavor, that to me becomes light animation to music vs very precise placement in the grid.  Room for both and with David's video macro he released for LSP you can do both very seemlessly in LSP.

I suppose I should figure out how to do the timing marks in audacity with it's tools, would shave a few hours off my LSP time.

David hasn't posted anywhere I have been for a few months.  LSP is way along in the re-write and it promisses to be a totally different animal when done.

There was a fellow working on an independent visualizer, I would like to see a 3d one of those  ;P

You know I tried LSP, I couldn't get a single light to blink in quite a few hours of playing with it. This was when i was looking for something that was more pixel friendly (vixen 2.0 at that time).. LSP was very frustrating, seemed to be quite powerful but nothing made a lot of sense. I then went to HLS and while that's another "complete re-learn" it worked and i had some success with it.. But nothing so far makes sequencing fun. I don't mind and enjoy listening to the songs over and over again, but no program has the ability to overcome my lack of artistry/music

Tory
Title: Re: an online sequencer
Post by: Frankr on December 10, 2014, 06:00:36 PM
First off I have to give credit to the originator of the concept sielbear.  He did a presentation a few years back at one of the conventions on the topic.  For the last three years I have done this with LSP and that spurred me going off and creating Light-elf.  Light elf made it easy for us to make huge sequences by stringing together 25 second sequences since LSP usually crashed with anything larger, after processing for more than an hour sometimes...

Last year dpitts also used this methodology.  The nice thing is it is very easy to share videos...

Here is Alan's (Sielbear) facebook page with some of his sequences.  A couple of years back he did a side by side video with the display and the video used to generate it.
https://www.facebook.com/liptonlights

Here is my facebook page:
https://www.facebook.com/rocklinlights

Now on to this year. with the addition of the whole house model to Nutcracker I was able to use the picture feature in Nutcracker. With ffmpeg you extract your videos into a sequence of jpegs at a rate of 20 per second. (ffmpeg -i <inputfile.mp4> -r 20 frame-%d.jpg)

You then just point the picture effect at frame-1.jpg and nutcracker will render each frame sequentially.  I think there was a check box added recently to force the picture effect to use 20fps and ignore the speed slider.  I am using a customized version that I forked a while back but you can still get the same effect.

Alan also switched over to Nutcracker. My render time went from >40 hours last year to roughly 40 minutes this year.  Major improvement!

Frank

P.S. I can share my frozen sequence and the video provided you can prove you own a copy but Alan would need to approve sharing the other videos.

Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 06:02:29 PM
Quote from: Livermore-Dad on December 10, 2014, 05:26:27 PM

You know I tried LSP, I couldn't get a single light to blink in quite a few hours of playing with it. This was when i was looking for something that was more pixel friendly (vixen 2.0 at that time).. LSP was very frustrating, seemed to be quite powerful but nothing made a lot of sense. I then went to HLS and while that's another "complete re-learn" it worked and i had some success with it.. But nothing so far makes sequencing fun. I don't mind and enjoy listening to the songs over and over again, but no program has the ability to overcome my lack of artistry/music

Tory

This. 100 times over. The biggest problem with all of the available solutions that I've seen/tried, is that regardless of how powerful the software is, or how many things it can do, none of them are easy to use, even for those like myself, who write and design software for a living. Some of them are so counter-intuitive and poorly designed (interface wise), they're almost unusable without hours and hours of tinkering.

I think this is primarily because the majority of the solutions out there have been created out of a need, by developers wiling to take a stab at it. And while many of these developers are extremely talented, very few, if any, have experience in usability, user experience design, or actually designing software interfaces. Most of the time, features are implemented in a way that's easiest or that makes sense to the developer, but many times this is a skewed point of view since they have intimate knowledge of the program and how it works, or they use built in controls or cross platform gui tools/frameworks. I don't necessarily fault them for that, it's much easier and faster to do so as custom interfaces are hard and usually take exponentially more time than native controls to implement.


Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 06:09:32 PM
Quote from: Frankr on December 10, 2014, 06:00:36 PM
First off I have to give credit to the originator of the concept sielbear.  He did a presentation a few years back at one of the conventions on the topic.  For the last three years I have done this with LSP and that spurred me going off and creating Light-elf.  Light elf made it easy for us to make huge sequences by stringing together 25 second sequences since LSP usually crashed with anything larger, after processing for more than an hour sometimes...

Last year dpitts also used this methodology.  The nice thing is it is very easy to share videos...

Here is Alan's (Sielbear) facebook page with some of his sequences.  A couple of years back he did a side by side video with the display and the video used to generate it.
https://www.facebook.com/liptonlights

Here is my facebook page:
https://www.facebook.com/rocklinlights

Now on to this year. with the addition of the whole house model to Nutcracker I was able to use the picture feature in Nutcracker. With ffmpeg you extract your videos into a sequence of jpegs at a rate of 20 per second. (ffmpeg -i <inputfile.mp4> -r 20 frame-%d.jpg)

You then just point the picture effect at frame-1.jpg and nutcracker will render each frame sequentially.  I think there was a check box added recently to force the picture effect to use 20fps and ignore the speed slider.  I am using a customized version that I forked a while back but you can still get the same effect.

Alan also switched over to Nutcracker. My render time went from >40 hours last year to roughly 40 minutes this year.  Major improvement!

Frank

P.S. I can share my frozen sequence and the video provided you can prove you own a copy but Alan would need to approve sharing the other videos.

I had a feeling you were doing something with the picture effect. I poked around earlier and was messing with it, I figured you were converting the videos to .gifs, but when I tried that with a LSP transition clip I had it failed miserably. I'll have to give that a shot..

I'd love to see how the videos are set up. Not sure what you're referring to when you say 'provided you can prove you own a copy', a copy of what?
Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 06:28:09 PM
Quote from: Frankr on December 10, 2014, 06:00:36 PMLots of good stuff snipped..

P.S. I can share my frozen sequence and the video provided you can prove you own a copy but Alan would need to approve sharing the other videos.

Folks by proving he means that you own a copy of the video, so when he shares he is not , you are not infringing on any copyright or whatever they call it now.

Would be interested in seeing it Frank. I watched the vid from the other side, where he is using a mega tree  and the everything else supports what is happening on the tree. Really neat

Tory
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 06:34:20 PM
Quote from: Livermore-Dad on December 10, 2014, 06:28:09 PM

Folks by proving he means that you own a copy of the video, so when he shares he is not , you are not infringing on any copyright or whatever they call it now.

Would be interested in seeing it Frank. I watched the vid from the other side, where he is using a mega tree  and the everything else supports what is happening on the tree. Really neat

Tory

Maybe I'm not understanding something.. own a copy of what video? I was under the impression that the videos being referred to are effects videos being created in video editing software. Like the video transition clips in LSP. Am I missing something here?
Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 06:36:25 PM
Quote from: tjames on December 10, 2014, 06:34:20 PM
Quote from: Livermore-Dad on December 10, 2014, 06:28:09 PM

Folks by proving he means that you own a copy of the video, so when he shares he is not , you are not infringing on any copyright or whatever they call it now.

Would be interested in seeing it Frank. I watched the vid from the other side, where he is using a mega tree  and the everything else supports what is happening on the tree. Really neat

Tory

Maybe I'm not understanding something.. own a copy of what video? I was under the impression that the videos being referred to are effects videos being created in video editing software. Like the video transition clips in LSP. Am I missing something here?

I guess they could be edited, but they are sourced from the real movie. So I think the part of the spiel is "reproducing in it's entirety or partial is protected under, blah blah blah" . so maybe that's the rub, where one is reproducing parts of a video and thus one needs to have the video to be in compliance with the various laws.  I'm guessing here!

Tory
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 06:42:32 PM
Quote from: Livermore-Dad on December 10, 2014, 06:36:25 PM

I guess they could be edited, but they are sourced from the real movie. So I think the part of the spiel is "reproducing in it's entirety or partial is protected under, blah blah blah" . so maybe that's the rub, where one is reproducing parts of a video and thus one needs to have the video to be in compliance with the various laws.  I'm guessing here!

Tory

Ha.. nevermind, I didn't realize he was talking about using the actual movie itself. I thought he was talking about using short video clips of color transitions/chases/etc and overlaying them on the models. I probably should've checked out both facebook pages ;). It didn't actually click until I looked closer at the let it go video and saw the tree. I'm not sure that would even work on my tree, I honestly don't think I have enough pixels for it to look right.
Title: Re: an online sequencer
Post by: Livermore-Dad on December 10, 2014, 07:05:15 PM
Quote from: tjames on December 10, 2014, 06:42:32 PM
Quote from: Livermore-Dad on December 10, 2014, 06:36:25 PM

I guess they could be edited, but they are sourced from the real movie. So I think the part of the spiel is "reproducing in it's entirety or partial is protected under, blah blah blah" . so maybe that's the rub, where one is reproducing parts of a video and thus one needs to have the video to be in compliance with the various laws.  I'm guessing here!

Tory

Ha.. nevermind, I didn't realize he was talking about using the actual movie itself. I thought he was talking about using short video clips of color transitions/chases/etc and overlaying them on the models. I probably should've checked out both facebook pages ;). It didn't actually click until I looked closer at the let it go video and saw the tree. I'm not sure that would even work on my tree, I honestly don't think I have enough pixels for it to look right.

I think the big win is that the entire display becomes cohesive, with similar colors, colors that work together etc. I'm going to give it a go,  but I don't have a ton of lights either, but even if it lit up my mini tree's and floods and strips with nice color, without me having to click lil boxes, that may be a big enough win!

Tory
Title: Re: an online sequencer
Post by: tjames on December 10, 2014, 07:09:11 PM
I'm curious about the settings. I had a copy of the frozen sing a long version that I ran through ffmpeg and grabbed the jpegs, but when I try to test them it doesn't seem to really do anything, I'm not sure what the movement setting needs to be either, unless it just doesn't work in the preview mode. Does the grid have to be set up in a certain way or is this something that can be added to an existing sequence with grid timings imported from an LOR sequence?

Title: Re: an online sequencer
Post by: Frankr on December 10, 2014, 09:19:43 PM
You need to check the box (though I don't know what it is labeled as I have not used the most recent code). This is so that nutcracker forces 20 FPS. I spent about an hour cropping let it go so that it was close up on Elsa's face most of the time.   This shows up great on my 32x32 grids.

The other sequences are just colors and effects from Sony movie studio so no copyright issues. My only concern is that they are Alan's IP and not mine. I'll check with him and share my Dropbox collection with everyone.
Title: Re: an online sequencer
Post by: Livermore-Dad on December 11, 2014, 12:22:49 AM
Quote from: Frankr on December 10, 2014, 09:19:43 PM
You need to check the box (though I don't know what it is labeled as I have not used the most recent code). This is so that nutcracker forces 20 FPS. I spent about an hour cropping let it go so that it was close up on Elsa's face most of the time.   This shows up great on my 32x32 grids.

The other sequences are just colors and effects from Sony movie studio so no copyright issues. My only concern is that they are Alan's IP and not mine. I'll check with him and share my Dropbox collection with everyone.

That will be appreciated Frank!

Tory
Title: Re: an online sequencer
Post by: Frankr on December 11, 2014, 10:21:43 AM
PM me your e-mail addresses if you want access to these and I will add you to the dropbox once I hear an OK from Alan.

Frank
Title: Re: an online sequencer
Post by: AussiePhil on February 09, 2015, 04:35:10 PM
Quote from: Frankr on December 10, 2014, 06:00:36 PM
First off I have to give credit to the originator of the concept sielbear.  He did a presentation a few years back at one of the conventions on the topic. 

Interesting, I know I did this in the middle of 2010 and credit where it is due, it was based on the original work from MPH with his original pixel megatree, his code ran on mac/unix and my scripting worked in windows and split the video file up into the relevant picture files and processed them for use in Vixen2, both my 2010 and 2011 shows relied on this concept.
All the effects in this video were generate in a video and translated into Vixen 2 data at 25ms playback http://vimeo.com/17761183

Title: Re: an online sequencer
Post by: Jrd on February 10, 2016, 11:14:48 PM
Did anything ever come of this online sequencer idea?
Powered by SMFPacks Menu Editor Mod