Unitalk with Doug Petkanics and Eric Tang: Scaling Livepeer to Thousands of Concurrent Video Streams
Originally posted on 20, Dec. 2018.
Live broadcast has become one of the most popular ways of mass entertainment. I believe that most readers have experience in live broadcast. So, when the blockchain technology is combined withlive broadcast, what kind of spark will it collide?
Livepeer is a completely decentralized, highly scalable, token incentive project that provides the infrastructure for a live video platform. What is the decentralized live broadcast? How do videos transcode to be compatible with different devices? What services can Livepeer provide for the live video platform?
Hi Doug & Eric. Nice having you here! Please introduce yourselves first before we get down to the hardcore stuff.
Doug: To introduce myself, I’m Doug Petkanics. I’m a co-Founder at Livepeer along with Eric. I’m a software engineer by background and we designed the Livepeer protocol together. Eric and I have been working together for about 8 years across two previous startups as well, in a founding/CTO/VP Engineering capacity.
We’ve been working in the blockchain space for the last 2.5 years, building mostly in the Ethereum stack, and had prior interest in Bitcoin for years before that. Great to be here. Eric, want to introduce yourself as well?
Eric: This is Eric — I grew up in Beijing, came to the US when I was 16. I’m a software engineer and entrepreneur based in New York, and learned about Ethereum in 2016. Really great to be here.
What is Livepeer? What problems are you trying to solve?
Doug: Livepeer is decentralized video infrastructure. The network leverages people’s idle compute resources to transcode video, aiming to dramatically drive down costs. This enables startups and scaled video applications to reach more users and use business models that were never possible with expensive cloud providers.
By comparison, cloud providers charge $3/stream/hour for transcoding which gets expensive when your application has 1000’s of concurrent videos. The costs in electricity is only $0.15/day to encode 2 videos concurrently. There’s a lot of room for disruption between the $0.15cents/day that idle hardware on the Livepeer network could charge compared to the $144/day that the cloud providers charge.
Eric: The Livepeer project is also open source video infrastructure beyond purely transcoding. There is an open media server that anyone can build upon, as well as early experiments in content distribution.
Comparing to traditional cloud video infrastructure providers, Livepeer also has the benefit of being decentralized, so there is no lock-in. Developers can now build video-enabled dApps.
What is LPT? How does it work?
Doug: LPT is the token that you stake in order to do work on the Livepeer network. You can either run a transcoding node yourself when you stake token, or you can delegate your stake towards another transcode node candidate and they’ll do work on your behalf. Essentially you’re either working, or providing quality assurance and security to the network.
Those who do the work earn the fees that broadcasters pay (in ETH), and they also earn inflationary LPT each day. This means that the only reason to hold the LPT is to stake it and do work — otherwise you are getting dilluted away, as the inflation only goes to those who stake.
However if you stake, you’re also providing security — if your transcode node cheats, then they and you can be slashed, and lose some of your stake. It’s this long term commitment that aligns everyone around trying to create the best, most useful network possible.
Eric: It’s important to recognize that fees are paid and earned in ETH (and eventually hopefully a stable token). Having LPT gives you the right to do and secure that work, to earn those fees.
It’s an ERC20 token, and it represents “the ability to provide service for the Livepeer network”. Since it’s being a service provider is a profit-making opportunity, there is an inherent demand for LPT.
It also serves as a coordination mechanism for an ecosystem to form on top of the network.
Token in the form of inflation is distributed to anyone who provides value to the network, and therefore funds the growth of the ecosystem. We call this “inflation funding”.
What are the use cases of Livepeer? Can you give us a few examples?
Doug: Any video application (centralized or decentralized) that uses transcoding can use Livepeer.
* Social applications like Youtube or Twitch
* Decentralized applications building new versions of these platforms could use Livepeer
* Government applications like traffic cameras
* Security industry applications like security cameras
* Industrial applications like self driving cars, that are capturing video every second they are on the road.
* Video touches almost every business, and the processing needs are growing dramatically with 4K, Ultra HD, 360 video, VR, and AR
I’m excited to see developers build social streaming applications (game shows, singing, chat, etc), using Livepeer’s infrastructure to save costs and scale.
Eric: It also enables some new types of video broadcasting on the web that was not possible before either due to pricing or censorship. For example, it has been too expensive for education-based broadcasting to exist (imagine universities broadcasting lectures to the web)
Online video is 80% of all the traffic on the internet, and the amount of traffic is expected to grow by 3x in the next 3 years.
And Livepeer can be used to encode/transcode the video, so it can be more scalable and affordable for the web.
What consensus algorithm do the transcoders use?
Doug：For quick background, we never really defined what “transcoding” is. But its when you capture one high quality video, say in 1080p HD, but it needs to be converted into many different formats and bitrates so that it can be delivered to mobile devices, computers and TVs with slower internet speeds, etc. The process of doing this is called transcoding, and video developers need to do it to reach a wide audience.
Consensus on the value and stake is provided by the underlying Ethereum Blockchain and its PoW consensus. So Livepeer does not need to come to Consensus in the way that a layer1 blockchain does, because that is provided by Ethereum.
However within Livepeer, Transcoder set election is determined by a delegated stake based system, which comes to consensus each round on the active set of nodes for that round.
Token holders stake towards the nodes that they want to be active, and then once a day the active set is locked in for that day based upon who has the most stake. After that, the smart contracts provide the rules of the system.
It’s important to note that the Livepeer nodes form a network OUTSIDE of the Ethereum network, so when video is being sent / processed on the network, it’s not going through Ethereum. The protocol minimizes the amount of data needed for each node to “be in consensus” with each other.
Centralized live video broadcasting is pretty mature right now. Why would anybody give up traditional ways and turn to decentralized live video platform?
Doug: The decentralized infrastructure will likely be cheaper to use if we are harnessing idle resources via a competitive marketplace. See the example I gave in the intro where a cloud provider may charge $144/day for 2 streams, but on Livepeer’s open marketplace a provider with an idle GPU may not need to charge more than $0.15/day to break even.
You may need to access extra capacity beyond what you’ve provisioned in a centralized way, in order to scale.
You may want to free yourself from having to run and operate your own infrastructure, or engage in a long term business relatinship with a centralized provider.
Eric: Centralized broadcasting works well because of the platforms provide a lot of easy for users. However, this comes at a price of censorship and lack of options for users. It’s almost impossible to switch to new platforms for broadcasters, and user privacy is not usually respected on these platforms.
A lot of people find out about Livepeer after some bad experience with a centralized broadcaster platform. While Livepeer is not a broadcasting application, we do hope there will be many applications built on Livepeer to address this problem.
Broadcasters pay the network in ETH for transcoding services. Is ETH the only payment method at the moment?
Doug: At the moment, yes. But it would be great to add a stable coin. This would allow people to price their services on the network in a stable currency that doesn’t fluctuate as much as ETH.
Eric: I think in the long term, payment payment method should be decided by the service providers in the network.
If the existing big live stream platforms build their infrastructure on top of Livepeer, would you be able to handle that kind of huge traffic and demand?
Doug: Typically in crypto networks supply always vastly exceeds demand, as you have competing actors trying to earn inflationary token prior to significant work and fees. As per our “video miner” proposal, which unlocks GPU transcoding, it would only take a couple mining farms to enable support for Livepeer to handle all the streams of the big social platforms like Youtube and Facebook. Bandwidth would become more of a constraint however. https://medium.com/livepeer-blog/the-video-miner-a-path-to-scaling-video-transcoding-a3487d232a1
Today’s network, as deployed on Ethereum, provides a nice working proof of concept, but is a little smaller by design while the platform is iterated upon. We have a scaling proposal out called “Streamflow” which proposes some architecture changes that will allow the network to scale to these levels.
Eric: We are currently working on “Streamflow”, which is kinda like Livepeer 2.0. It addresses some of the issues around scalability in the current version via mechanisms like probabilistic micropayment.
Decentralized live video streams are censorship resistance — it is an advantage indeed, but is it possible that people would make use of it to do illegal things like broadcasting some restricted stuff to the underage?
Doug: This still depends on the application being built, and access points for distributing the content. Under the hood the LP network is just infrastructure for transocding, and currently the applications that are built on top still need to distribute their content. They could leverage other decentralized tech projects like IPFS, Swarm, Ethereum, etc to create censorship resistant applications, but out of the box Livepeer doesn’t give you a full application platform.
If users sent bytes of data through the network that represented malicious or illegal content, any node operator could choose to moderate and not encode it theoretically, but this would be akin to blaming TCP/IP and the routers on the internet for malicious bytes of data being routed across the internet.
It’s still an important question that we think about every day. We want to give node operators and developers the right tools to ensure that they are operating in a way they are comfortable with.
Eric: If we look at how centralized streaming platforms handle this, we realize they just develop internal tools so abuse can be detected quickly and removed from their application. I think tools like this can exist in Livepeer, so that application builders can use them to monitor their own content.
Is your goal to overthrow the centralized video platforms or to cooperate with them (by persuading them to use your infrastracture)?
Doug: The centralized platforms should adopt the Livepeer protocol and run its open source software in order to power their own internal transcode newtorks, and then fall back to the public networks when they need excess capacity, or rent out their own idle capacity to the network when they are not at peak utilization.
In the end, it’s about providing the world’s most efficient allocation of computing resources, driving down costs for everyone, putting more money in the hands of the content creators and users, enabling everyone to participate in the world economy, and creating amazing video enabled applications that otherwise couldn’t have existed during centralized cost structures.
Eric: I think a lot of video platforms will co-exist. Users who want to use centralized video platforms should have the options to do so, and I think platforms like Netflix have a place in the world and are unlikely to become decentralized in the short term. However, there are A LOT of use cases for video, and we believe many of them can be better served in a decentralized context.
Do you know the Thunder? It operats like the more people using it to download one video, the faster the speed gets. Does Livepeer work the same way?
Eric: I’m not familiar with it, but it feels familiar to the mechanisms in Bittorrent, which is a p2p file sharing protocol developed in the 90s by Bram Cohen. Livepeer is definitely inspired by that work. It’s similar in the sense that spare capacity is being shared for everyone, therefore the network becomes more performant as more people join the network.
We haven’t really touched on the p2p video distribution side, which is in research phase now. The p2p distribution mechanism will allow viewers to share their bandwidth when they are watching the same video, therefore making the download faster for everyone.
Will those transcoded videos be stored somewhere? Now many live stream platforms support replay.
Doug: Video transcoded by Livepeer can be stored at any location depending on the broadcaster’s preference. Currently the network supports IPFS and S3 (we are adding Google Cloud soon), but it’s very easy to add new storage methods. After the video is stored, it can be used for playback.
I don’t know about Twitter or Youtube but in China there many live stream platforms which are completely free — neither the broadcaster nor the user has to pay any fees. Why do you say that centralized platforms are expensive?
Doug: Platforms that are free for users are actually the perfect users of Livepeer. Under the hood they are subsidizing the costs of infrastructure for their users. They are either paying a cloud provider or are paying for people/machines to run their own operations.
This has very high costs, and as a result they need to make it up with their business model — either advertising/virtual goods/etc that effect their user’s experience. If their infrastructure was much cheaper, they could pass that value on to their users through better experience (less ads), or even share more of the revenue with the users (streamers/performers/etc)
Video consumption nowadays are really for leisure purposes. So you have to make sure your product can be acquired very easily and consumed instantly. Is Livepeer easy to use?
Doug: Livepeer is a developer infrastructure. Developers who are building “very easy to use” applications, can do so and use Livepeer’s infrastructure. But under the hood you interface with Livepeer protocol directly using developer API’s, cryptocurrency, and other more complex technical topics. This should all be hidden at the app layer of course — just like when users use a streaming service, they don’t need to know anything about the underlying APIs that power the video streaming.
Eric: Livepeer is as easy to use as any other cloud service providers. There is also an opportunity for someone to create the “Coinbase of Livepeer”, which is a service focused on bringing Livepeer to anyone who doesn’t want to use crypto.
There are various formats of videos and different models of devices to watch them. How can you make sure that the transcoded videos are compatible with all these?
Doug: The open source Livepeer Media Server supports the most popular formats — H264 video served over HLS. However it is easy to add additional formats as they are required. Anyone can submit a pull request, and often times its simply a matter of wrapping the correct arguments to the underlying video libraries.
Eric: Livepeer currently supports h264 at any bandwidth. This covers over 80% of the devices out there. It’s also very easy to add new video codec into the Livepeer stack since the entire stack is over source. Similar to how ffmpeg ends up supporting every video format, Livepeer will do as well.
How can those who don’t have any tech background participate in your network?
Doug: Good question. The best way is to play the role of delegator. You review nodes past performance and campaigns and decide whom to stake tokens towards and route work towards. You’re essentially QA’ing the network, and they share their fees back towards you.
Many of these nodes are campaigning on additional contributions they are making, such as building community, building tools, etc — so its also possible to get involved in the more community or business side of the project.
Eric: There are a lot of token holders who don’t have a tech background. One simple way to participate is through staking. There are also other opportunities like helping to grow the Livepeer ecosystem. If you are interest, you should join the Livepeer wechat or discord group to learn more.
Does Livepeer have any dApps? Is Livepeer TV your product?
Doug: Livepeer.tv, crypto.livepeer.tv are both sample DApps that were developed in collaboration with the community. There were also one off dapps for events like devcon4.tv, and embedded LP streams at ethberlin.com. These are all what I would call “pre-scale.” They support one off event streaming. But as we deliver the Streamflow proposal, we can focus on getting scaled video applications with many streams to use Livepeer’s infrastructure.
Eric: There are a few dApps that already use Livepeer — for example, paratii. Livepeer TV is a community project created by some Livepeer community members, focused on streaming Ethereum events and Berlin-based artists.
[The copyright of this article belongs to Unitimes. Please contact us at firstname.lastname@example.org if you want to repost the article. Opinions expressed by contributors belong to themselves.]