• NBC Universal Hackathon – Miami 2018

    This weekend I went and spent some time hacking some code at the “NBC Universal Hackathon” and trying it out new ideas, meeting new friends, and learning a ton on many technological aspects and innovating.  The particular problem that we decided to to solve was the irrelevance  aspects of current TV and how more interactive could it be with current technologies.  The way to solve it thru a collaborative experience where users can interact with their phones and cameras with the video shown on screen.

    The team was composed by: Satya, Paul Valdez, Juan Gus, Myself, and Chris.

    What we did what simple, we could create a website that could have a canvas that could be treated with effects, add the TV/Video feeds into it and  that distribute the content using a platform like “Cloud to Cable TV” to cable operators or OTT/IPTV systems.

    Cloud to Cable TV

    The solution required a few items to be setup and configured:

    • RTMP Server or WebRTC Setup to receive video feeds from Smartphones or your laptop,
    • FFMPEG to encode, compress, and publish  video/audio feeds
    • Mobile App with RTMP Client or WebRTC Client or laptop. We tried several but this one worked out ok.
    • A web application in Python to map each feed and position it on top of the TV Channel video source (assuming an M3U8 feed or a movie in MP4)

    With this in place, it is a matter compiling CRTMP, FFMPEG, and we tried other components as Deep Learning such as the “Deep Fakes” project. The idea that we had was to replace one of the actors image, as well as superimposed our live feeds into the video.

    Issues:

    • The safari browser doesn’t allow you to play content with autoplay features, meaning that the user MUST initiate a playback. If SAFARI sees or detects that onLoad the content autoplays this fails.
    • There are issues with SAFARI and dynamically loading the content and video.oncanplaythrough() is required to be added to the javascript.

    The live feeds had a delay of about 30-40seconds, as it had to:

    • Convert and push from mobile phone to RTMP Server,
    • Grab RTMP Stream and send it as an m3u8 encoded file to the website.

    The standard CRTMP Screen would look like and connections from Gus and Pablo successfully took place:

    
    +-----------------------------------------------------------------------------+
    |                                                                     Services|
    +---+---------------+-----+-------------------------+-------------------------+
    | c |      ip       | port|   protocol stack name   |     application name    |
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 1112|           inboundJsonCli|                    admin|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 1935|              inboundRtmp|              appselector|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 8081|             inboundRtmps|              appselector|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 8080|             inboundRtmpt|              appselector|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 6666|           inboundLiveFlv|              flvplayback|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 9999|             inboundTcpTs|              flvplayback|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 5544|              inboundRtsp|              flvplayback|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 6665|           inboundLiveFlv|             proxypublish|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 8989|         httpEchoProtocol|            samplefactory|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 8988|             echoProtocol|            samplefactory|
    +---+---------------+-----+-------------------------+-------------------------+
    |tcp|        0.0.0.0| 1111|    inboundHttpXmlVariant|                  vptests|
    +---+---------------+-----+-------------------------+-------------------------+
    

    We were trying to use WebRTC but we had many issues with latency and delays.

    The FFMPEG command required and that was used for the demo was:

    ffmpeg -re  -i rtmp://96.71.39.58/live/pablo -c:v libx264 -c:a aac -ac 1 -strict -2 -crf 18 -profile:v baseline -maxrate 200k -bufsize 1835k -pix_fmt yuv420p -flags -global_header -hls_time 3 -hls_list_size 4 -hls_wrap 10 -start_number 1 /var/www/html/live/pablo.m3u8
    
    ffmpeg -re  -i rtmp://96.71.39.58/live/gus -c:v libx264 -c:a aac -ac 1 -strict -2 -crf 18 -profile:v baseline -maxrate 200k -bufsize 1835k -pix_fmt yuv420p -flags -global_header -hls_time 3 -hls_list_size 4 -hls_wrap 10 -start_number 1 /var/www/html/live/gus.m3u8

    The Mobile App published an RTMP Stream to the server under /live/pablo and /live/gus.  The demo video on what it will look like:

    https://vimeo.com/299048743
    Screen capture in Vimeo using Safari

    For screen capturing in a Mac with FFMPEG with 3 screens, list your devices and capturing to avoid any MOOV issues and useless MOV/MP4 files.

    ffmpeg -f avfoundation -list_devices true -i "" 
    
    ffmpeg -f avfoundation -i "3:0" -t 120 -pix_fmt yuv420p -c:v libx264 -c:a libmp3lame -r 25 teleport.mov

    The presentation we made to the judges at the “NBC Universal Hackathon” can be found here:

    https://docs.google.com/presentation/d/1sKAvnC-Y-KHu2qclulH2Fp-8yWvTslq6bLaocyEgtfQ/edit?usp=sharing

    The source code consists on an HTML site using DOM objects, video source, and a canvas. As shown, the video is hidden it is native format in ways that you can use canvas drawing to copy the video from the “src” in m3u8, MOV, MP4 or whatever format your browser can handle to the canvas. The canvas is then the placeholder for all the overlays and divs. The idea with the canvas is that messages can then by typed and exchange between users, as a WhatsApp application or any other chat application that uses the canvas.

    var canvas = document.getElementById("c");
    var context = canvas.getContext("2d");
    
    window.onload = function() {
     // document.getElementById("fb-profile").style.display = "none";
      
        var canvas = document.getElementById("c");
        var context = canvas.getContext("2d");
        // grab the video element
        // var video = document.getElementById("v");
        
        // drawVideo(context, video, canvas.width, canvas.height);
        // calls drawVideo() function, passing all the objects
    
    }
    
    var splayer = {};
    
    function showIt(id, url, hideOrNot) {
      console.log(id+"  "+url+ " setting it to " +hideOrNot); 
    
      splayer["v_"+id] = document.getElementById("v_"+id);
      document.getElementById(id).style.display = hideOrNot;
      if (document.getElementById(id).style.display == "none" ) { 
         document.getElementById(id).style.display = "block";
         var vId = "vsrc_"+id; 
         console.log("playing "+vId + "  "+url);
         document.getElementById(vId).src = url;
         if (splayer["v_"+id].paused) { 
            console.log("Video paused.... ");
            splayer["v_"+id].load();
            splayer["v_"+id].oncanplaythrough = function() {
                splayer["v_"+id].play();
             };
         } else {
           console.log("Video is playing already..."); 
         }
      } else {
         console.log(" Stopping .... v_"+id);
         splayer["v_"+id].pause();
         document.getElementById(id).style.display="none";
      }
    }
    
     var player = document.getElementById("v");
     
    function ChangeHarry(){
        console.log("Playing Harry Potter.... ");
        document.getElementById("vsrc_main").src = "http://s3.us-east-2.amazonaws.com/teleportme/videos/HarryPotterClip.mp4";
        player.load();
        player.play();
        drawVideo(context, player, canvas.width, canvas.height);
    }
    
    function ChangeQueen(){
      console.log("Playing Queen of the South ... ");
      player.pause();
      document.getElementById("vsrc_main").src="http://96.71.39.58/queen0.mp4";
      player.load();
      player.play();
      // drawVideo(context, player, canvas.width, canvas.height);
    }
    
    setTimeout(function() {
           showIt ("first", "https://mediamplify.com/teleport/iwantharry.mp4", "none");
           setTimeout(ChangeHarry, 6000);
         } , 2000 );
    
    setTimeout(function() { 
          showIt ("first", "https://mediamplify.com/teleport/iwantharry.mp4",  "block"); 
    }, 8000 ); 
    
    setTimeout(showIt, 5000, "second", "http://96.71.39.58/live/pablo.m3u8", "none");
    setTimeout(showIt, 6000, "third",  "http://96.71.39.58/live/gus.m3u8", "none");
    console.log("Starting changing stuff"); 
    
    setTimeout(function() {
                console.log("Preeping to switch to Queen of the South" ); 
                showIt ("first", "https://mediamplify.com/teleport/iwantqueen.mp4", "none"); 
              }, 13000);  
    
    setTimeout(showIt, 15000, "third",  "http://96.71.39.58/live/pablo.m3u8", "none"); 
    setTimeout(showIt, 15010, "second", "http://96.71.39.58/live/gus.m3u8" ,  "none"); 
    
    // setTimeout(showIt, 20000, "third", "http://96.71.39.58/live/gus.m3u8", "none"); 
    setTimeout(function() { 
                console.log("Queen of the South");
                ChangeQueen();                        
                showIt("first", "", "block");
               }, 19000); 
    
    
    
    function fbProfile() {
        var x = document.getElementById("fb-profile");
        if (x.style.display === "none") {
            x.style.display = "block";
        } else {
            x.style.display = "none";
        }
    }
    
    function drawVideo(context, video, width, height) {         
       context.drawImage(video, 0, 0, width, height); // draws current video frame to canvas     
       var delay = 100; // milliseconds delay for slowing framerate
       setTimeout(drawVideo, delay, context, video, width, height); // recursively calls drawVideo() again after delay
    }

    For a functional demo, 1st allow the site to play video in autoplay:

    Update your settings in SAFARI

    We didn’t win the “NBC Universal Hackathon” but had a ton of fun doing it!.  We failed in the presentation, it was only 3 minutes and our presenter disappeared in the last minute, and Gus improvised and didn’t use all the time provided by the judges. We knew we were done when no questions were asked. …. Anyways!!! You cannot always win.


  • Music Choice Patent Axed Partially by PTAB

      

    Late September and early October total of two judgments have been filed thus far by the Patent Trial and Appeal Board (PTAB) in the case against Music Choice’s patent. As indicated in the rulings for IPR2017-00888 and IPR2017-01191, partial axing fo the patents have been granted to Stingray or against Music Choice’s interest.  Interesting findings that enforce the value of our cloud to cable TV patents.

    The final written decisions are available herein:

    As Law360 points out:

    ” The case split the PTAB panel, with each of the three judges filing opinions. The majority ruling by Judge Mitchell Weatherly held that eight of the patent’s 20 claims are invalid because Stingray showed that a person skilled in the art would be motivated to combine the earlier inventions to arrive at Music Choice’s invention, which includes both a playlist and on-demand playback. Source: https://www.law360.com/media/articles/1091845/ptab-partly-axes-music-choice-patent-in-row-with-rival “

    This goes in accordance to the prediction made by myself back when this issue was discussed initially, specially in the article where I analyzed the acquisition of Music Choice by Stingray digital.

    Infringement and Damages Model

    As expected Music Choice should be more interested not so much on the “Video on Demand” revenues which is what these two patents cover, but more so in the patent covering the method and system for broadcasting to Cable TV which is the main source of revenues for both companies. The patent in dispute will be resolved this Friday October 19th, 2018, and the patent 8,769,062 as well as the original applications from 2001-08-28 which is claimed as priority date.

    It is my opinion that a similar outcome may result from the PTAB panel.

    Hence, what would the damages be for Music Choice? Well, I don’t have access to any information but what is found online, and in a letter from 2014, Music Choice was claimed to have 56M homes, or 57M listeners per month.  Source: https://www.justice.gov/sites/default/files/atr/legacy/2014/08/18/307851.pdf ‘

    In this document, Music Choice indicates that their Multi-Video Programming Distributors (MVPD) constitutes a main source of income for the company. Additionally, in a marketing material VOD distribution is said to have 72M subscribers as of Music Choice’s 2016 PDF :

    http://corporate.musicchoice.com/files/2214/6228/4671/2016_Music_Choice_Media_Kit.pdf

    In a similar filing, Music Choice makes a case with a rebuttal testimony by Gregory S. Crawford, PhD https://www.crb.gov/rate/16-CRB-0001-SR-PSSR-SDARSIII/rebuttals/2-21-17-music-choice.pdf 

    Hence, assuming a price point of let’s say 0.05c to 0.25c/subscriber and recalling what Mr. Boyko said in a prior-conference call, Music Choice should have revenues could be:

    • 0.05c per subscriber @ 50M subs = $30M/year
    • 0.25c per subscriber @ 50M subs = $150M/year

    Assuming now that, Stingray has landed $8M/quarter with an increase of 102.2% or 23.5% o the total revenues, from the 4th Quarter Report available online

    ” … or the quarter, Canadian revenues decreased 2.6% to $13.6 million (41.3% of total revenues) due to decrease in non-recurring revenues related to digital signage, United States revenues increased 102.2% to $7.8 million (23.5% of total revenues), whereas revenues in Other Countries increased by 34.4% to $11.6 million (35.2% of total revenues). (CNBC source: https://www.cnbc.com/2018/06/07/globe-newswire-stingray-reports-fourth-quarter-2018-results.html) “

    What this means is that somehow per year $32M/year o revenues were eroded from Music Choice’s pie, or since 2014 when the lawsuit was said to have had some issues with Stingray, then the total loss could be $32×4=$120M on damages.  Which, it seems to have been the offering made by Stingray digital to Music Choice for a full acquisition.

    The question is, what would be the forward revenues would be for a Georgia-Pacific model that a damages expert would have done for pricing moving forward and how Stingray Digital would have to pay for any future earnings to Music Choice. The information collected indices a 35% tax by Stingray on royalties and potentially another 20-30% which in turn would make operations in the United States at no profit or even at a loss.


    Conclusion

    It is very hard to compute a damages calculation, assuming that infringement takes place in the surviving claims of the patents in dispute. Hence, if damages were to be computed should be in the range of 8-9 digits with upside to the future, not including potential treble damages which could account for more money to be paid.

    Alternatively, Cloud to Cable TV  is the best technological platform for monetization with Cable TV and works using the most recent advancements in Cloud computing, Web technologies, and in combination with standard DVB Systems.  A unified patented technology for Cable TV distribution!

    https://eglacomm.net/cloud-to-cable-tv/
    Cloud to Cable TV  – US Patent 10,123,074