With the evolution and the increase that we see every couple of years in term of the bandwidth reserved for our internet connection either on our desktop or mobile devices, the needs of the users evolve and adapt to that change, starting from just consuming lightweight and "text based" content, to more interactive and heavy ones like videos (YouTube, Twitch, Dailymotion etc ...).

I had a couple of projects i did just for fun during some occasions, just to improve my skills set on the video and multimedia part, since the first time i worked on such projects was a while ago ... (If you were a fan of Anime during your teenage years, you would try at least once to translate some Japanese subtitles to another language).

But in the recent months, i noticed that Facebook changed their whole video system, in order to compete with YouTube and gain some market shares by giving more visibility and organic reach to the videos uploaded there than the ones on other video sharing sites.

Facebook made a huge success with video content:

Facebook would have reached $17.45 billion in revenue last year, surpassing its $12.5 billion milestone from 2014 by 40 percent.

[...] Global spending on video ads grew by 41 percent from third quarter to fourth quarter, according to data from Nanigans, a Facebook marketing partner.

Source: Facebook Inc (FB) Q4 2015 Earnings Preview: Mobile, Video Obsession To Drive Company’s Success

Besides the growth that we see in the current time, the expected increase of video content on mobile devices in the upcoming years is not negligible, some forecasts say that 72% of mobile traffic will be video content.

BY 2019, 72% OF MOBILE TRAFFIC WILL BE VIDEO CONTENT [CISCO STUDY]

Viral videos on Facebook & Automation

My obsession to know how things work, and how some pages on Facebook became famous within a very short time just by using videos (Unilad, SoFloAntonio etc ...), made me think: "Does the page owner really open his video editor and edit the video's title, resize the canvas of the original short clip and add his page's watermark every single time ?" ...

The answer might be yes or no ... but my inner decision tree pushed me to say: "Let's challenge this and automate it".

Operation name "Automator"

Just before getting into technical details, just take into consideration that i won't discuss the setup process neither how to type "apt-get install" or "yum install" ... It's just an overview about the architecture of the system and how i implemented it, so it's not really for newbies (Sorry for the Clickbaiting :P).

Automator: Code name "Alpha" (I like fancy words too)

This is just a simple version of the system, i started with this just to create a small PoC.

WebApp : Back-office

The back-office is used to manage the whole inventory of the videos (I used just a SQLite Database for development purposes).

Amazon S3

Used just to store the assets that will be used with the editing, also to store the uploaded and the converted/edited videos to be downloaded later.

Message Broker

Since the process of converting and editing the video is too much time & ressource consuming, it's better to create asynchronous tasks/jobs that will be managed by a messaging broker (ex: RabbitMQ, Apollo ...) to dispatch the load across workers and synchronise between them to handle the tasks needed to be done.

Workers/Converters

I won't talk in detail about the different tasks needed to be executed by those workers (Download, Conversion, Edition and Re-upload), but just take into consideration to separate each task with a specific queue to organise the workload and to make each queue easier to debug (Best practice).

Automator: Code name "TripleS (SmallSaaS)"

Just a small improvement you might want to think about, is to add a hub that manages the webapps by granting them the wanted permissions to communicate with the broker, with credits support if you want to for example.

An optional CDN might help too, but it's not really necessary since latency is not our main concern here, because the results that we will get won't be delivered in a continuous way and to huge amount of users directly from the platform.

"Automator" is alive !

Some screenshots of the beast (Just the back-office part):

Pretty satisfied with the results

Watching a young boy running over his older brother might be priceless sometimes (The watermark is just used for testing purposes):

If something is not clear or if you have any further questions or inquiries, just let me know in the comments section or via e-mail [email protected] .