Categories
IT Networking Streaming

Another Type of Re-Stream Server with Node.js

I came across another type of re-stream server that might help some people. It can also be run on a $3.50 per month server from Amazon Lightsail. It’s a node.js server called Node-Media-Server roughly according to this link: https://github.com/illuspas/Node-Media-Server. I’ve modified it slightly and made it useful for re-streaming to YouTube and Facebook.

The reason I searched out yet another re-streaming server type is that I found that streaming with Wirecast going to my other server under NGINX was unreliable. That server works very well with our hardware streamer and the streaming apps on iPhone and Android but it didn’t work well with Wirecast. The following server works very well with Wirecast as well as our hardware streamer and the apps that I have on my iPhone and Android phone.

My version here, is slightly different than the one on Github and I’ve laid out some instructions below. Please read the entire post before starting as we want to make sure it goes as smoothly as possible.

The first step in the process is to to to Amazon Lightsail. The website address is : https://aws.amazon.com/free/compute/lightsail/ They even have a one month free trial of their services for the smallest one. That’s the one we want so it’s great just to try for this. So even if you don’t know what you’re really doing, you aren’t even out anything to try.

Go to Get Started Now and begin. You’ll have to create an Amazon AWS account to start, but that’s easy. (I bet you’ll like it after the first month too) I can’t go back through this because I already have an account. I don’t think I can walk you through some of those details because of that.

However once you are through with that, you should get to a page that looks like this:

From here you choose your instance location. I always choose Northern Virginia for two reasons. One is that I’m located 40 miles from these server farms. Two is that these are probably some of the fastest server farms in the world.

Next you have to pick the OS for the server. Also you have to choose “OS Only” and I choose Linux with Ubuntu being the image I want. It runs the software amazingly and the instructions assume these choices. Now we have to name our “instance”. I like something like “re-streamer” or something of the sort.

Next you have to wait for it to be created, that takes about a minute or less. Once it’s up and running, we can log into it via the browser. That is very simple and you just click the hyperlinked name of your server. That brings you to a page with a huge button that says “Connect Using SSH”. That’s what we want.

Then it brings you to a command prompt. This is where we’ll issue all our commands to make the server do all our work.

Then you can begin with the following code at the command prompt. Press “y” when it asks and select “yes” by putting the red curser over it when you see a graphic ask a question. Or if it asks to press [enter] please do that.

My directions below are slightly different than the ones on Github but these solve some little issues you might find if you followed the others.

The below commands you should perform one at a time by cutting and pasting into the command prompt.

mkdir nms
cd nms
sudo apt-get install npm
npm install node-media-server
sudo apt install software-properties-common
sudo apt update
sudo add-apt-repository ppa:jonathonf/ffmpeg-4
sudo apt install ffmpeg
sudo nano app.js

When you use “sudo nano app.js” you’ll be opening a file. This is the main configuration file for the node-media-server. You’ll need to paste in the following code but put in any locations you want to send your stream to via RMTP. You have to replace my keys with your keys obviously.

const NodeMediaServer = require('node-media-server');
const config = {
  rtmp: {
    port: 1935,
    chunk_size: 60000,
    gop_cache: true,
    ping: 30,
    ping_timeout: 60
  },
  http: {
    port: 8000,
    allow_origin: '*'
  },

  relay: {
        ffmpeg: "/usr/bin/ffmpeg",
        tasks: [{
                app: "live",
                mode: "push",
                edge:"rtmp://127.0.0.1:1936/rtmp/<<YOUR FB KEY>>",
                appendName: false
                },
                {
                app: "live",
                mode: "push",
              
edge:"rtmp://a.rtmp.youtube.com/live2/<<YOUR YT KEY>>",
                appendName: false
                }
               ]
        }
  };
var nms = new NodeMediaServer(config)
nms.run();

We have to get stunnel so that facebook will work. We’ll go back to the command prompt from your server. Issue the following commands to get stunnel.

sudo apt-get install stunnel4 -y

Now we’ll have to change stunnel’s boot configuration, issue the following command:

sudo nano /etc/default/stunnel4

Change Enabled from 0 to 1. It should look like the following:

ENABLED=1

Next we have to edit the stunnel configuration file.

sudo nano /etc/stunnel/stunnel.conf

You’ll have to cut and paste this in its entirety. It should look like this:

pid = /var/run/stunnel4/stunnel.pid
output = /var/log/stunnel4/stunnel.log
setuid = stunnel4
setgid = stunnel4
# https://www.stunnel.org/faq.html
socket = r:TCP_NODELAY=1
socket = l:TCP_NODELAY=1
debug = 4
[fb-live]
client = yes
accept = 1936
connect = live-api-s.facebook.com:443
verifyChain = no

Then of course you’ll have to use ctrl-x to exit, and Y to save it as the original named file.

Next we have make it enabled after boot by doing the following:

sudo systemctl enable stunnel4.service

Now we have to restart stunnel because we changed the configuration files.

sudo systemctl restart stunnel4.service

Now that Stunnel is installed and the app.js file has been properly created we can try it out. To make it run you have to execute the app.js file. Use the following command. The & after the command will let it run in the background and won’t close when you close the command prompt

node app.js &

This should output something like the following:

Whenever you have to modify anything in that app.js file remember that it’s located in your home folder, under “nms”. So in linux you have to go to that location from login like this if you want to directly:

sudo nano ubuntu/nms/app.js

Then remember that to start it again you have to use the command “node app.js” or “node app.js &”.

Also if you have to stop the app.js from running you can press cntrl-x, which will exit the process and give you a command prompt again.

You’ll want to fix the firewall settings on the lightsail server instance so that our connections from outside can get to the server. You should allow ports, 8000,443, and 1935 along with the ports that are automatically allowed by lightsail. You can change this by going to the “networking” section of the instance. Then making it look like the picture below. You’ll also want to set the IP address of your streaming location so that other people can’t stream to your instance. the instance I created for this tutorial isn’t going to be running by the time you read this. I’ll be deleting it, so it will be safe.

Next you can test it with some simple software like VLC along with OBS or your churches streaming software or hardware encoder. You’ll need your RTMP feed to go to your server IP address which is seen on the home page of the instances of Lightsail. It’s on the bottom right above your server location. You can also set it to make it static at this point, but that will give it a new one that will not change. You’ll want that so you don’t always have to change your OBS or streaming software or hardware. To make a static IP just go to the instance by clicking the hyperlinked name and then choose networking from the options across the middle of the page. Then choose static IP on the left.

This gives a nice output page as well to see the status of your running server that’s waiting for your stream. Go to your IP address 000.000.000.000:8000/admin

In OBS or your hardware encoder choose these options in the stream settings:

Service: Custom
Server: rtmp://<<ngnix server IP address>>/live
Stream Key: test

For a playback test use VLC or something that can play RTMP streams. Set it up as follows: Go to Network Stream, Network.

network URL: rtmp://<<node-media-server IP address>>/live/test

You can also go see what the admin page looks like at that IP address:8000/admin, it will show you the inputs and outputs as well as memory usage and cpu usage.

Also now you should be able to go to Facebook and YouTube to see it there to go live with if you wish. Everything should be up and running at this point. You might want to take a snapshot here, because that way you can always fix anything else that goes wrong. Also if you want to further modify this for other purposes to make it better later you have a good jumping off point to create a new instance without jeopardizing your running one.

******The below section about PM2 is an addition to this post to make the program work better for streaming events without worrying about the terminal window closing or disconnecting.

To make this app.js function continue to work even if you close the command prompt, which you will probably want to do reliably. You should install another small program that will make this continue in the background and continue pushing your streams anytime you need them. We’re going to use PM2 to do this.

Install PM2 by typing the following at the command line:

sudo npm install pm2 -g

There are some huge benefits for you if you run your application using pm2. Make sure you are in the directory /nms/ where you installed the app.js file, and instead of running the app as we did above (with node app.js), we’ll run it now using the following command:

pm2 start app.js

This will show something like:

This PM2 kept our server up and running without any input for a 24 hour test we inadvertently ran yesterday through this morning.

Also remember that to type “exit” to get out of a terminal window safely. Don’t just close the window with the X on the top.

If you need more help, please email me tim@timdrones.com or find me on Facebook.

Categories
Computers IT Networking Streaming

Statistics and Webpage for the HLS server

I need a few statistics for the HLS server because as you stream the .m3u8 file, it’s a bit hard to tell how many people are using the app to watch it. I’ve inserted a couple statistics pages on this server to help get a better pictures of who’s using it.

The first way is to use the RTMP streaming statistics as seen in the /etc/nginx/sites-available/default file at the location at the bottom.

## XML stylesheet to view RTMP stats. Copy stat.xsl wherever you want and put the full directory path here
location /stat.xsl {
	root /var/www/html/;
	}
## This provides RTMP statistics in XML at 
location /stat {
	rtmp_stat all;
	rtmp_stat_stylesheet stat.xsl;
	}

This outputs a nice little page that looks like below:

The next bit of Stats that I added was the below. This is from the stub_status output. It’s very simple and only somewhat useful, but it’s a start on seeing how many people are on my .m3u8 feed. The active connections is the useful part however because of the internal connections and my own viewing of this page, out of the 11 seen connections I think only about 3 or so are actually users on our app in the below case.

I put in a little index.html file to replace the original that tells you Nginx on a standard install of Nginx. I made my page just give a few links to the various status reports on the server. It makes it a little bit easier to use and will help when I train others to use these servers later. The status page looks like the below:

This is made with an index.html file with the links to the low resolution, high resolution HLS feeds as well as the two status pages.

  • Connection link: https://your_domain_name_here.xyz/nginx_status
  • RTMP feed status: https://your_domain_name_here.xyz/stat.html
  • Low Res link: https://your_domain_name_here.xyz/low.html
  • High Res link: https://your_domain_name_here.xyz/live.html

I’m still trying to find more ways to get better statistics out of the HLS feed use, however I haven’t gotten there yet. I’ll probably keep this updated to tell others when I find more ways to do it.

Categories
IT Networking Streaming

My HLS Server

The second server took a lot of work to get correct. It was much harder but the work payed off and it’s functional. I’m still trying to upgrade it to get better statistics about how many people are using the app to watch live but I have some going now.

Server configuration for restreaming and modularly sending to HLS receiving apps.
We use this server in the above configuration. In ours, the RTMP streaming server is now a node.js node-media-server according to https://timdrones.com/2020/05/12/another-type-of-re-stream-server/

To make this part function, the server takes in an RTMP stream and converts it using ffmpeg to make it an HLS .m3u8 file. That file is then the target of a link in the apps control panel that tells it to get the stream on a users phone and display our live service. Each user of the app who watches within the app is connected to the HLS streaming server and so we have to scale this one up to use it well.

I have a snapshot of the server that is in AWS Lightsale and when we need it I can spin it up to be what would cost $40 a month. But we only need it for a couple hours. Then we delete it. So it costs about 12 cents to run each week for our live service on Wednesday. I just transfer over the dns records and IP address information and allow the proper ports on the firewall and in 5 minutes it’s ready to go.

I also found data about statistics by searching for “stub_status Nginx” on Google. That led me to information such as: https://www.google.com/amp/s/easyengine.io/tutorials/nginx/status-page/%3Famp

The configuration is similar to the RTMP re-streamer in that they both use Ubuntu and NGINX and this one needs ffmpeg to do its conversions of the RTMP feed into the HLS feed for the app.

I used a tutorial I found online to begin this one as well. It’s found below.

https://www.ustoopia.nl/featured/nginx-rtmp-hls-ssl-videojs-on-ubuntu-18-04/

After going around and around trying to recompile Nginx with the stub_status module, I found the tutorial that told me how to check if it was already installed. It turns out it was. So that was good however I wasted a lot of time trying to install it and a c compiler when I didn’t need to.

If you’d like to see my NGINX.conf file for the HLS streaming server an example of it is here. If you’d like it to also stream to other services you can use it, but you have to modify it a little bit like the RTMP server that I posted about previously. Facebook only accepts “secure” rtmp so you have to pass it’s feed through “stunnel”. Here’s my nginx.conf file. I also put in the “Streamingserver.xyz.conf” file that you will need in the sites-available folder on the server. This file should be named after your proper registered domainname and .com or .org or .xyz or whatever your domain name is. The default file below is “default” as seen in the sites-available folder on the server as well. These are all in “/etc/nginx/nginx.conf” or the others are in “/etc/nginx/sights-available/”. If you follow along in the examples that I gave above from the links to outside sites, you can see that you’ll make a link between the sites-available folder and two files in the sites-enabled folder with the same names of “default” and “yourserver.something.conf”. Also while I was creating this, I had to stop the apache server because it was interfering. I had to issue the command “sudo systemctl stop apache2”. It was causing errors.

If you’re interested in the app we use it’s The Church App from Subsplash.

Categories
Computers IT Networking Streaming

Virtual Services and Streaming

This is what our server configuration looks like. This is a quick document that I made up to show the leadership what I did.

Because of our new uptick in streaming we had to figure out how to stream our live services with RTMP to Facebook and YouTube, as well as our app. The app is a little odd as it requires a stream input of a .m3u8 file, also called HLS. Our church cable modem is not able to support more than one 1080 x 30fps output at once. This is about a 4mps stream. So we needed something to help us.

This led us to research services like restream.io and Boxcast. Both these will do it for RTMP, however Boxcast will do that and also the .m3u8 (HLS). That brought up an interesting challenge in my mind. I figured that this couldn’t be “that” hard. I just had to figure out how to re-stream RTMP. That led me to cloud servers which I had a tiny bit of experience with after-all this website is on an AWS Lightsail cloud server which costs $3.50 and my other website timrides.com is on a free Google cloud server instance.

I had experience in spinning up cloud servers with Bitnami originally and running apache as a webserver. I had seen the wonders of what can be done, but didn’t know how to do more than run WordPress on those Bitnami installations for the websites.

So I went to Amazon and to the Lightsail section which is sort of the simple part of AWS where you pay for various size cloud servers and all your data and things are included. It makes billing simple and you don’t have to worry much about causing some huge bill. You can get there at https://lightsail.aws.amazon.com/ .

The first part of making the re-streaming server for RTMP is getting an instance such as the $3.50 one, it’s good for at least re-streaming to 3 places. So you take in your RTMP feed from your streaming software like wirecast or obs and you send it to the server. The server takes it and sends it on to the services like YouTube and Facebook. If you need more, you might need a bigger server like the $5 one. In any event its cheap and even if one is to small you can take a snapshot of it and spin up a bigger one in about 4 minutes. I chose to run Ubuntu 18.04 on mine. It’s well documented and easy to run NGINX on that to get a good webserver and robust re-streaming server all in one.

I did a ton of Google searching for information and came upon some reasonable tutorials. It was not very hard to get the re-streaming part of this going. Just forwarding streams of RTMP is not to hard and obviously doesn’t even take much computing power since the Ubuntu machine with 512mb of RAM can do it. It requires some simple config file modifications to the NGINX software with the rtmp module in it.

Also to get Facebook to work you have to add in a module called “stunnel”. That is pretty simple as well. The tutorial below should have most of what you need for Facebook, YouTube and any other service that requires only RTMP.

We’ve now streamed a few times to Facebook, YouTube and my HLS streaming server a few times. It works very well with this simple RTMP streaming server setup. It’s running well on my tiny $3.50 per month server just chugging along. Once the persistent stream keys are input in the server there isn’t any maintenance for it or any other reason to look at the configuration files. I’ll modify it sometime to add some of the statistics information that I have on my HLS server.

In addition to the below, you should fix the firewall to only allow through ports 1935, 80, maybe 440,990 and 1000. Depends on what else you do with it.

https://sites.google.com/view/facebook-rtmp-to-rtmps/home

If you’d like to see some of my nginx.conf file and other parts needed for this project see below. I’ll insert them here.

If you need to output an HLS or .m3u8 file, you need to go a little further. I made two servers so that I could get the majority working quickly and then I could modify the other without messing up the working one.

My second server needs to be made bigger to carry the load when we stream to the app. The app users are directly receiving the file from my server and they put a load on it. Also the creation of that HLS file is intensive alone, even if only one person is on the app. I’ll post about that second server in another post.

I hope this brings you as much of a feeling of accomplishment as it brought me. It also taught me a ton about linux and cloud servers.