The Fine Art of AI Upscaling

iSize Technical Articles

By Sergio Grce, CEO, iSize Technologies


My background in Economics and Finance, including more than a dozen years in the investment banking sector, began to go stale in 2014. I needed a challenge. I needed to test myself in the real world with real products. I wanted to be directly involved in developing something new and innovative that could change whatever industry I chose, rather than be limited to moving money around.

Fundamentally, I wanted to apply the skills that I had developed in earning my Masters Degree in Finance with what I had learned in my investment banking career, but in a different way.

The way I chose was to get involved with some very clever technical people to develop a technology that can potentially change an entire industry, if not the world to some degree. I mean, if you’re going to dream, you might as well dream big!

What brought me to where I am today was the result of fortuitous encounter with just such clever technical people, who were using deep learning algorithms to mimic the style of master painters to reproduce their style for consumer images. It was a very intelligent way of fusing fine art with digital art, which resulted in a unique art form that rendered favourite photos with the painting techniques and styles of Van Gogh, Monet, Picasso and many others.

However, that was quite a few years ago, and the computational power required to do it made it very complicated, time-consuming, and expensive. The artificial intelligence and deep learning models we were using at the time were slow and not really useful for anything else, like for example App development.

We were therefore pretty limited and used what power we had to print and sell online the art that our algorithms created. But we were something of a victim of our success. We soon discovered that because the end results were so compelling, customers were ordering larger and larger prints. But we weren’t able to upscale the images with the degree of resolution and quality required using the existing software and algorithms available in the market at the time.

So we dug deep into our expertise to find an answer, and that answer was to develop our own upscaling algorithm, and it turned out to be very good indeed. In fact, it was so good that people were soon using it for image rendering and upscaling to virtually any size, and doing it very quickly. That was the moment we realised we were onto something, and that what we had developed had multiple, dare I say limitless, applications. At the very least, our technology could vastly accelerate a great number of routine, and often frustrating, imaging tasks.

We created a company, iSize, to further develop, exploit and now, market what we have created. The upshot is that it almost immediately became apparent that an extremely logical place to apply our expertise is in an industry that remains heavily reliant on processing increasingly large volumes of the data that make up those images… in this case, moving images.

So, what we were doing with still photography rapidly evolved into applications for video, but this time taking advantage of the computational power of the cloud. Our business model for what was at the time a still somewhat nascent 4K market emerged from the concept that it would be a good idea to apply our algorithm to create 4K content by upscaling existing HD content.

After all, it’s no secret that there are staggeringly large repositories of content around the world being digitised every day for archiving, often in hopes of one day being repurposed. We obviously knew that film or SD/HD content cannot be reshot in 4K or 8K, but it can definitely be upscaled.

Of course, the majority of modern televisions upscale HD to 4K every day. “Full HD” TV’s are nearly extinct on the high street. But the quality of 4K can often be underwhelming. All can just about get the job done, but some sets upscale better than others. However, between the half-dozen PhD’s at iSize, we’ve come up with something that we believe is not just better, but ground-breaking. So why not offer our brand of AI powered upscaling to a market that can really use it?

And that’s precisely what we’re now doing, supported by our team of video signal processing and machine learning experts devoted to not only improving the quality of the visuals we provide to multi-platform devices, but streamlining the transfer of really large video files, anywhere.

It’s important to state here that our bitrate saving technology (“BitSave”) is independent of any codec.

Yes. You read that correctly. We do not break any standards with our bitrate saving precoding technology, which means it can be an add-on solution to anyone’s existing codec infrastructure. Our codec-independent technology means that we can save up to 70 percent in bitrate requirements in video delivery systems without waiting for new codec standards. It also makes encoding up to 500 percent faster because a smaller pixel footprint is entering the encoder.

What that codec independence means is that we can approach any industry and easily test our technology with them. It could be defence, VoD streaming, production, hardware, e-learning, e-Gaming, VR/AR, IoT… you name it. If you are streaming high quality video content, we can help optimise its movement to any viewing device, all done via our SaaS platform which we will be launching during IBC 2019.

Right now, we’re small and a lot of people don’t yet know us, but being under the radar initially is not necessarily a bad thing. We learned early on that the development and standardisation of codecs can take time, often quite a lot of time. And that means a lot of people have to agree.

We quite deliberately made our technology codec independent. To make it work, we can develop and present our working ideas and innovative technologies to people very quickly. We currently have more than 10 trials taking place with some very large companies worldwide, and the results are looking very promising indeed.

But even more pleasing is that in addition to the favourable feedback we’re receiving, we’re also learning from every single client interaction we have. Those who are potentially going to use our technology are educating us just as much as we are sharing what we’ve learned with them, and we’re collectively putting all that information to very good use, discovering exciting new applications almost every day.

So, even though it’s already a genuinely exciting time for iSize, it’s only the beginning of our journey and what we are planning to bring to the market in order to change the way the high quality video content is delivered to the viewers, now, rather than in the future.

There’s an art to it.