Media and entertainment is one industry generating and moving vast amounts of data at the edge. This episode of Utilizing Edge brings Jimmy Fusil of Tsecond into a discussion with Alastair Cooke and Stephen Foskett about the unique data challenges in media and entertainment. There is a great need for local processing, secure data transport, and efficient storage utilization in managing the ever-increasing volume of data generated during film and TV production. The podcast explores the continued relevance of physical media, like tape, for long-term archiving, while also emphasizing the security measures necessary to protect creative privacy and prevent potential plot leaks in the industry. Fusil also discusses Tsecond’s groundbreaking product, Bryck, a portable NVMe storage device capable of holding up to a petabyte of data with a remarkable 40GBps data transfer speed. The discussion highlights how Bryck’s high-performance storage facilitates on-device data processing, ensuring quick backups and maintaining data integrity. It’s critical to consider edge storage solutions like Bryck in many data-intensive sectors beyond media and entertainment as well.
Hosts and Guest:
Stephen Foskett, Organizer of the Tech Field Day Event Series, part of The Futurum Group. Find Stephen’s writing at GestaltIT.com, on Twitter at @SFoskett, or on Mastodon at @[email protected].
Alastair Cooke, independent analyst and consultant working with virtualization and data center technologies. Connect with Alastair on LinkedIn and Twitter. Read his articles on his website.
Jimmy Fusil, M&E Expert and Business Development Manager at Tsecond. You can connect with Jimmy on LinkedIn and learn more about Tsecond on their website.
Follow the podcast on Twitter at @UtilizingTech, on Mastodon at @[email protected], or watch the video version on the Gestalt IT YouTube channel.
Transcript
Stephen Foskett: Welcome to Utilizing Tech, the podcast about emerging technology from Gestalt IT. This season of Utilizing Tech focuses on edge computing which demands a new approach to compute, storage, networking, and more. I’m your host Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. Joining me today as my co-host is the fine Mr. Alastair Cooke. Thanks for joining me today Al.
Alastair Cooke: Thanks for inviting me sir. It’s always a pleasure to catch up with you and it seems like we’re going to be doing a bit of a Back to the Future finding that everything old is new again today.
Stephen: I think that’s true. Those of us who’ve grown up in it and been here for a while are familiar with the proverb of the station wagon loaded with backup tapes or the sneakernet or the challenges of moving data wouldn’t you say?
Alastair: Yes I think we’ve all gotten sort of a different experience over time but it’s fundamentally, it’s always been that moving data is a problem and as we solve the problem of moving a certain amount of data, the amount of data that needs to be moved increases and we’ve definitely seen over the last few years the expectation of the speed at which you can move things and you can respond has really increased and so we’re seeing some real, real challenges around moving around large amounts of data and enabling business agility through that.
Stephen: And one area that there is a lot of data of course is media and entertainment so we know that video production is increasingly, well increasingly probably predominantly digital this time and of course we know how big these video files can be whether it’s the Utilizing Tech podcast series or watching videos on Netflix or the the videos we’re producing on our phones. There’s a lot of data there and a lot of that stuff is recorded on location all around the world. That’s what we’re going to be focusing on today. So one of the things that we haven’t really discussed yet on Utilizing Tech is the world of media and entertainment and that’s our focus for today. So we have invited Jimmy Fusil M&E expert and business development manager for Tsecond to join us. Jimmy, welcome to the show.
Jimmy Fusil: Thanks Stephen, hi Alistair. It’s great to be here and thank you for having me. I look forward to talking about all these problems that we have in the media and entertainment industry.
Stephen: yeah it’s a cool industry.
Jimmy: yeah, I’ve been working in this industry for 20 years and yeah, data has you know, through that transition, like you were talking about, to digital and as usual you know, as you said, you solve a problem and then that problem grows and you have to solve it again. So that’s why I’m working at Tsecond and I think it’s what they have is a is a potential solution to not just now but the future of moving that data around for the media and entertainment industry.
Stephen: Yeah it is an amazing industry. I’ve been to NAB show for example and seen the amazing sort of parallel world, parallel universe of technology that they have. Like Alistair said, what’s old is new again. There are all sorts of specialized devices and specialized approaches for this, but Jimmy why don’t you start by just talking a little bit like what does a modern production look like in the field?
Jimmy: Well at the core of it is a camera right, and a microphone or several cameras and several microphones, and those cameras on a typical movie or TV set in the US create three to five terabytes of data every day and you’ve had this cluster, this crew right, these people have come together for let’s say six months to make this TV show and they all bring their own preferred gear their own preferred device and they all try to share this data amongst them and then have to move this data to various processes you know, from transferring what the camera filmed into a smaller file that the editor can then edit and then giving that same file that the camera shot and giving it to visual effects or several visual effects companies to add some magic to it and then finalizing the picture and the image and then moving all that to be localized so that it can be distributed around the world in every language. It’s a lot of data and it’s big data a very high resolution, very high quality and it takes a lot of work just getting things where they need to be. That’s the biggest thing and you know, the surprise is you know, while networking has brought great convenience to a lot of this, we’re still you know, using I don’t know if anybody else uses this term, I’m in my little Hollywood corner over here, but sneakernet is that still you know, that’s still a thing for the media and entertainment industry.
Alastair: I’m interested how much of that is transferring from the site of recording and how much is transferring between different devices that are at the location where the recordings made because there’s a whole different set of challenges around that speed of movement. Is this about moving between the different favored devices at that edge location where the video is being produced or is it about moving between the different contributors or both of these things?
Jimmy: So there is data moving amongst devices on the set, so a compressed video is moving from the camera to a monitor that the director can watch and the producers can watch. That’s being recorded in the playback system so that if the director wants to see something they shot two hours ago they can pull it up, but that’s a different set of data than the data that’s being recorded into the camera that will be used for post-production. There’s all this metadata that’s flying and actually it can be a challenge on movie sets to have the amount of bandwidth and to manage also the interference of all these different networks that are where with all these different devices communicating and then if you get into more advanced production technologies such as virtual production where you’re playing back video on to super large LED system to act as as a fake background let’s say, now you’ve got a bunch of networked devices that are playing back and managing the LED system and also enabling correction to those images are being projected. So you can have actually a whole set of networks on site of various devices communicating or interfering with each other on the set. But that’s one challenge and usually that is smaller data. The bigger challenge as far as big data right, is getting this this video that’s coming out of the cameras and getting it to its next phase which is QA syncing with audio and creating files that the editor can then use to start editing the movie or the episode. So both things.
Stephen: You know, some might say is this really edge but I think that it absolutely is. It definitely meets my definition because it is bandwidth constrained, it is availability constrained, meaning that you sometimes can have outages. It is also the province of people who are not IT people which is another similarity with retail or industrial you know. In other words these are people who are trying to get their job done and they’re not all that interested in why isn’t the network working or how do I, you know, optimize this. They just want their camera to work and you know, so I think that it definitely is and I think that this gives us a whole like opens a window into a different world, but one that other areas can learn from because if we think about it, this kind of data is present in other areas as well. I mean I don’t want to get off topic but I could see similar discussions in oil or exploration. I could see similar discussions in industrial IoT and manufacturing in autonomous vehicles, basically anywhere that you’ve got sort of sensors out there in the real world, whether they’re cameras or lidar collecting lots of data that then needs to be moved around and coalesced. But let’s focus on the M&E space. So you know there’s moving data from the on-site of production back to the studio, but as you talked about there’s also moving it to New Zealand so that you know, what I can look at it right and Alastair and all these other production houses. I don’t know if you’ve watched movie recently but the ending credits, there’s more than two units now and there’s a heck of a lot of production studios involved in these movies right?
Jimmy: Yeah that’s true. I mean it’s, movies have always been a collaborative process right where many people come together to make a movie or a TV show and now that that collaboration is global across time zones and across data centers and moving you know, sometimes it’s only 300 gigabytes of data to a production house or VFX house in in New Zealand and sometimes it’s you know terabytes of data that’s been captured because a volumetric capture of a set, for example where that needs to be reproduced and digitized. So yeah it has to be in many places and like you said you look at at the credits and it’s not usually one or two companies working on these movies it’s tens to dozens you know? So it’s a lot of shoveling data around and that’s really happened over the last 20 years of digitization. First you know, files started to matter because we were digital now instead of filming on actual film and then the practices became more complex so a lot more files were created, different types of files, and then the files got bigger because the cameras now film at 8K, 6K, and have greater bit depth so the files themselves become bigger and now they have to be all over the place and they have to be reliably mimicked in all these different places because the production itself, the set will travel, the post-production team will travel, and the VFX are happening in all these different localities whether it’s Europe or India or New Zealand or Canada. So it’s a big data movement and data custody problem.
Alastair: I think that was one of the interesting challenges for this as an edge to me is that the location where you’re actually operating is transient the the on-site production. I mean here in New Zealand, there is a famous tour of going to all of the places where they shot video for the various Lord of the Rings and Hobbit films in New Zealand and they’re all over the place and Jimmy I’ve got to expect that the production crews are expecting the same service no matter where they go now but just because you’re in in the back of beyond on the top of some beautiful hill you can’t say well there’s going to be a three-day delay before you can see any of the results of what we’ve done today.
Jimmy: Exactly sometimes that processing has to happen on site you have to drag along a miniature data center to take those files out of the camera and take that audio and do what you need to do so that it can be watched in what we call dailies or rushes. The next day you know every everybody used to sit in the theater and look at it together now everybody’s looking at their laptop and usually looking at it from a cloud-served service right? So you’re processing locally uploading the results and then watching it back locally via cloud service of some sort. And yes, network connectivity remains super important and yes, sometimes it’s not enough to move the data to where it needs to be processed. It needs to be processed on site and that to me is also fitting the definition of of edge for sure and then we can look at you know, what I actually think are these kind of mobile edge processing devices which are you know, broadcast trucks right where you’re going to an event, filming it, collecting all this data, and you’re already starting to process it because you’re already starting to edit it, the director is already picking their favorite angles and they’re already doing some color adjustments and such. And then you know, that broadcast truck packs up and it moves to another event and moves to another place. I think of those as also you know really edge processing kind of systems as well.
Alastair: And there’s another aspect that you keep hitting on is that the actual data files, the original files shot by camera are way too large to do any of the the real-time fast processing on, so there is a sort of effectively sampling of that data to get a lower resolution, lower color depth version of the film that’s used for these dailies for making, even being sent to the editors and I think there’s an interesting analog to when we’re generating large amounts of sensor data, we may need to very rapidly summarize that data in order to be able to make real-time decisions. And that’s essentially what’s going on. We’re getting a summary of the video data being put in front of humans who are making fast decisions and that’s definitely a pattern we see in a lot of the edge deployments and it seems to have been built into the cameras to actually do this as I understand not something that’s done after the capture of the video.
Jimmy: That’s true certain cameras can create a proxy file and upload it right away and that’s you know, there’s this vision of camera to cloud and it works you know, you can see that file almost instantaneously once somebody says cut and if there’s plenty of bandwidth you’ll see that file somewhere else in the world. But the actual high definition you know, high resolution full color depth file getting that up and into the cloud and processed, that’s still something that’s being worked on and certainly there are productions that are doing that with a lot of resources, but it’s still in its infancy for sure
Stephen: And I believe that you have a solution for that too. So I think it would be fair to to say how do you do that?
Jimmy: Well, yeah, thank you. I love talking about the problems. So let’s, thank you for reminding me to start talking about maybe some of the ways we can solve it. That’s really what’s interesting about the work being done at Tsecond with this Bryck device which is really a way to get massive amounts of data from the collection point into the cloud or to process it locally because it really is capable of both. So I’ll describe it a little bit. It’s a brick right it’s nine and a half by four by four inches and it’s a device that’s basically an NVMe storage device that can hold up to a petabyte in certain configurations like in that in that block right? And it connects to a server via this tray with the proprietary connectors and the tray right now is rack mounted and you can get 40 gigabytes per second on and off of it, 40 gigabytes. I always have to remind myself to say bytes and not bits. And so I like to think of it both ways you know where we talked a little bit about sneakernet earlier in some ways it’s a way to sneak your net a whole bunch of data very quickly because it can ingest data very quickly through that connection and then it can move very securely because it’s very rugged. It’s, there’s proprietary process of bonding all these drives together so that you can drop it, you can leave it in water for a few months, you can take it back out, it still works. And so very rugged form factor and you can travel with that securely because it’s very temper proof so you can move this high high value data in that way very quickly because sometimes it is still faster to to walk or drive or fly with it than it is to to put it through the network. But the other thing that’s really interesting about it is that you can with that kind of performance on and off the drive in that portable factor you can really build a mini, it’s almost like a mini data center. You add some compute to it, you add some workstations around it, people can you know like I said collaboration is key in making a movie or TV show, so you can point a whole bunch of people to that data and they can work in this highly collaborative way locally right, and in that way I think of it like you know we talked about like as an edge device in many ways because it enables you to do that high processing work locally before you take the data somewhere else.
Stephen: So from a technical perspective essentially it’s a like a giant thumb drive except much much much better in every way. But you know, but I think in terms of sort of conceptually understanding it I imagine that audio video production engineers and so on they look at it probably like the removable media that cameras use and I don’t know if people are aware but yeah a lot of cameras have adopted or the same sort of removable media that your own camera might use except they use different formats some of them use SSDs you know like regular SATA SSD, some of them use much more advanced stuff but at least they conceptually can understand this brick right here is like a piece of media that my camera can write to and I can use it that way. Is that how they approach it or is it approached as sort of a different animal entirely than removable media?
Jimmy: It’s approached in some ways in that way like we did a proof of concept with a camera manufacturer where we pointed 20 cameras, sorry 10 cameras we could have done 20 but we only could get 10 and recorded simultaneously like it was a common you know SSD for all these cameras and they were filming at 8K and 60 frames per second so you can imagine the amount of data coming off of each of these cameras. So that’s a way of thinking it that is analogous to what you’re you’re talking about, Stephen, which is let’s look at it as a portable storage right but there’s you know, there’s companies that are very very impressed by the performance in that small package, the fact that you can get 40 gigabytes a second off of this Bryck through this tray and the whole system really enables them to parallelize a lot of processes that in the past when you pull that card out of the camera and then you put it into the card reader, the card reader connects via USB or if you’re lucky Thunderbolt and you get those files off of there and onto another piece of storage that then has to, you know, from which you do all the other processing you need to do. So there’s opportunities both in capturing directly to the Bryck and there’s opportunities also in processing directly from the Bryck and taking all these steps out which you know, it’s copying data so in some ways it’s very simple. In other ways it can be very scary because this is where things go wrong, this is where you know, the drive gets misplaced or the truck rolls over it or you know more than anything you know, there’s corruption or because you’re not in a great environment for computing you’re you’re going into a place that’s either too hot or too dusty or all those things and and so minimizing those errors is actually and minimizing also the time between the camera you know, stopping record and getting access to those files is a big thing for productions these days for sure.
Alastair: And I’m sure there’s a security angle on there as well because if you’re making the the latest Marvel Blockbuster, there’s a lot of fans who would be very keen to see some of that content long before the studios are ready for it to be released. Is there, you talked about the robustness and the tamper-proof nature, are there also security capabilities built into the product?
Jimmy: Well there’s definitely some integrity, data integrity software that really looks for a bit flip and corrects it constantly so that data integrity piece but I think from a data security perspective, it’s a bit more what you’re talking about is just that the fact that’s a proprietary connector that you need this tray to connect to, you can really, it makes it very difficult for people to access this data. You have to figure out you know, you would have to own a tray basically and a server and therefore you would have to be in some ways a customer of Tsecond and to be able to access this data. So I think that’s, you know, you can’t pull it apart either just physically like you pull it apart, you destroy it right? So that makes it very secure and you know, I will say that like like you point to Alastair, they’re like the industry is very very very sensitive to data security because a leak means that the surprise is gone, the leak means that the creative privacy of that director and the actors working together to make something is violated. So it’s taken a while for this industry to even warm up to the cloud you know. We’ve been banking there much longer than we’ve been making movies there and there’s still I think this remaining feeling of safety and security and having something that’s physically hardened as well which is very familiar to to this industry.
Alastair: So we used to the idea of server huggers from the virtualization days and now we have, I guess, it would originally have been tape huggers but the movement away from tape and towards much more non-linear media has made a big difference. But yeah holding on to that the physical media seems to still be a big big cultural piece there.
Jimmy: Yeah with, since you bring up tape, I think tape is an interesting medium because it is still very dominant format and I’m not talking about videotape. I’m also talking about LTO, you know, archival tape. It’s a big part of how the M&E industry in the production the M&E industry and the production phases leverages tape to archive, but also to create backups. There’s a real desire to have multiple copies of the footage as soon as possible, so that if something happens in one location, one copy of the footage, there’s a backup somewhere, there’s a data integrity somewhere and this industry has gotten very good at writing tape very quickly but then accessing that data becomes a very big problem. And so I think in some ways, we’re getting to the density of tape and even higher density, but we’re also you know, with the brick with t secondsBrick we’re getting to a point also where you have that density and that security but you also have that performance which has been missing from tape in some way since it’s maybe because M&E is so reliant on physical media like you said but that is, that’s a level of security that I think that the industry is attached to.
Alastair: And of course the multiple copies is a learned lesson from, I think it was one of the Toy Story movies where an administrative error in the data center destroyed the only recording and the only way the video of the movie actually got produced was that somebody was on maternity leave and had a backup on their own or a copy on their own workstation. So yeah it’s definitely a lesson that could have been a very expensive mistake. I think the different media to solve different sort of problems so using things like tape as long-term archive where the cost of retention, the cost of storage is the most important thing versus the speed of access I think this there’s probably also a piece around the retrievability of the individual parts. I’ve heard some discussions around making new video you’ll often use segments out of a previous video that maybe may have been produced 10 years ago and the retrievability of the individual segments is really important and I think that’s where non-linear media like the Bryck becomes much more usable than tape media.
Jimmy: Yeah it is, it is hard to exercise those archives it is hard to exercise a tape archive sometimes because you are looking for, you can’t really scrub through it right? You have to restore a whole bunch of things and then hopefully you’re very organized, but I also you know, I also think that tape has its place in the sense but using it as a transfer medium I think which is what we’ve sometimes seen in this industry around post-production just because of ruggedness and its security, that really to me is something that it’s not always super well adapted to.
Stephen: Yeah I imagine too that some of these folks yeah your tape huggers might be really wanting to have a copy on tape as well as on the Bryck, and I imagine if it’s attached to a server, I imagine that’s possible right? I mean you can basically coalesce everything on the Bryck and then dump it to tape before you give it to the production assistant to fly across the ocean with right?
Jimmy: Yeah I think you know actually that brings up another proof of concept we did with a post-production company and we’ll have a white paper coming out on that in the coming month um where you know the process there is they received the cards or the, sorry, the footage from the camera and copy it to their system and that starts a chain of of processes which includes syncing audio, adjusting color, framing and creating the dailies, but also archiving and in the past because of the limitations of the performance of their storage they had to work through these things linearly so you copy, you ingest the data, you validate it, do you know, a hash check on it, and then you move it to syncing and then you color it and then you create the deliverables and then you start the archive. Well in this case with the Bryck, when they put it in, realize we can start archiving right away once we do the validation we start archiving and there is it’s first, it’s nice to be done archiving earlier but also there is this feeling that okay, now we already are starting to make the backup copy, so if something disastrous happens we know we’ve already started that process right? And that I think goes to the point that this performance on this storage is also very necessary because you need to do a whole bunch of things all at once with this data right away.
Stephen: Yeah and also you know we talked about the sneakernet, another colorful IT term is shoe shining, that you have to keep tapes streaming if you want to get the performance because tapes actually do have quite a lot of bandwidth, but only and when they’re when you’re really pouring the data in there and when they can get the data and they don’t have to stop and start and reset. So I imagine that having you know a nice high performance source for that data is important as well to make sure that you can actually make use of tape media. So I mean the whole thing, the whole production process sounds, it’s really amazing, it’s really interesting to learn about how these things happen. I guess kind of in conclusion, in summary, you know, Jimmy what’s your message to the edge about this technology, whether they’re in the media and entertainment space or not?
Jimmy: Well I think you know, the most precious moment in media and entertainment is that collection moment right you need to collect that data and then you need to hold on to it and get it to the places where it needs to go as quickly as possible and to me that’s kind of true, that’s kind of what the paradigm of edge is telling us is like really important things are happening right there right and they need those those processes need to be serviced and then that data becomes incredibly precious. And so I think the message is like you know find a way to get it as quickly as possible, find a way to make it safe, and then find a way to get it out of there and to the places it needs to be and I feel like you know sometimes that’s going to be a network, but sometimes it might not be because that location might be aa jungle or it might be a desert or it might be a poorly service warehouse somewhere downtown and you just need to get that data where it needs to go and make sure it gets there safely. So maybe let’s think it’s almost like network is now the box and I think there’s you know there’s other, there’s other ways maybe let’s maybe it’s time to think about sneakernet sometimes again if it makes sense.
Alastair: Jimmy I think I’ve learned something really significant from you and that’s is that my expectation that edge locations are about refining data and getting the most value out of the data by making it less. It doesn’t actually match every use case here clearly the high resolution, finest granularity biggest volume data is the most valuable thing that you’re producing the summarization and the the smaller units of data that rushes dailies, the lower resolution versions being sent to editors those are a little less valuable than the full volume data. So again it just shows the variety of challenges that organizations face with different use cases out at the edge.
Jimmy: Yeah I think that’s true and I think we’re seeing that the collection of data and the maintaining that high quality data is becoming important in new ways as well. In the entertainment industry, this and work there’s now even more interest than there’s ever been before in 360 video and 180 video and immersive what we can put under an umbrella of let’s say immersive video right? And a lot of that is captured from multiple cameras or giant cameras that have even more resolution and you’re right, that sometimes you want to refine that data right away but really in the media and entertainment industry and the production mindset you’re trying to collect as many options for yourself as possible that you may utilize later. You want that flexibility and so you don’t want to let go of anything. You might process things so that you can look at it, so you can make decisions around whether it was a good take or whether that is the right costume or whether that was a great performance but you want to hold on to all of it because it has value in ways that you may not realize yet and that is I think maybe I think that I’m kind of echoing what you’re saying, so I think the data is about getting processed locally but it’s also about sometimes preserving all of it and all of its potential value that we may not realize it has yet.
yeah I think this has been really interesting because you know it’s a unique you know, M&E is a unique environment, it’s a unique edge environment, but it’s also similar to others. And I think it’s important for people to understand that there is technology that can be applied from one area to another and I, that sounds to me like what this is, it’s applying some pretty cool enterprise technology to a very you know, some remote jungle somewhere and maybe this makes sense for other applications as well, so I urge our listeners to check out Tsecond, Jimmy where can they do that? Where can they continue this conversation with you?
Jimmy: Well they can find us at Tsecond.us that’s t s e c o n d dot us and then they’ll probably be seeing a white paper come out in the coming month around some of our work with the industry, some application of Bryck there and you know on the website you’ll also see like you said that there’s plenty of other applications for this device that is already you know, being used in many other domains, so if you’re curious about it, that’s the best place to go.
Stephen: Alastair, what have you been up to in the last few weeks?
Alastair: Well I continue to slave away at my my day job but also organizing the VBrownbag Tech Talks to VMware Explore and I will be seeing you there Stephen in about is it three weeks, it’s getting awfully close. Yeah, It’s kind of amazing to think that it’s getting that close.
Stephen:Yes I’ll see you at VMware Explore. I’m also going to be at Flash Memory Summit this week and at a number of other events including Edge Field Day which is coming up. If you’re interested in learning more about that, go to TechFieldDay.com. So as mentioned, you can find me on social media at SFoskett. I’m here every Monday with Utilizing Edge, part of the Utilizing Tech podcast series. You’ll also see me on Tuesday in the On-Premise IT Podcast and on Wednesday in the Gestalt IT News Rundown. If you enjoyed this discussion, please do subscribe. You’ll find us in your favorite podcast applications or on YouTube please do give us a rating, please give us a comment, we would love to hear from you. This podcast was brought to you by GestaltIT.com, your home for it coverage from across the enterprise. For show notes and more episodes go to our special dedicated website UtilizingTech.com or find us on Twitter and Mastodon at Utilizing Tech. Thanks for joining us and we’ll see you next week.