Reason & FL Studio - cross platform performance benchmark

This forum is for discussing Reason. Questions, answers, ideas, and opinions... all apply.
Post Reply
User avatar
SH1audio
Posts: 4
Joined: 18 Jan 2018

18 Jan 2018

Hey, this is my first post here ;)
I'm SH-1, and I'm a FLeb mainly- I've been producing in FL for roundabout 11 years now. I've always liked tinkering with friend's and clients' copies of Reason though, and when VST support was added I really had no reason not to buy one for myself. I started a few tracks and the thing that irked me the most was... compared to FL Studio, I VERY quickly ran into the brick wall of 100% CPU usage on all cores. I mostly chalk that up to VST handling in Reason, since from personal experience there has been a bit more headroom when I used the native and RE formats exclusively.

So I got together with a Reason power user who had made a benchmark file using a lot of VSTs that we both have in Reason and I painstakingly transferred every single detail into FL Studio (it's harder than you might think), double and triple checked against the Reason file to make sure the MIDI interpolation didn't screw up the automation in FL and tried to match the colours from the Reason session :D

There is a single block version and a version with 6 copies of everything that stack up incrementally. Smart disable has been left OFF for the FL projects to ensure a fair comparison.



Here are the single block results:


Reason document in idle:



FL Project in idle:



Reason Document playing:



FL Project playing:



Now, on to the 6-layers project:

Reason document in idle:



FL Project in idle:



Reason Document playing (CPU usage max set to 95%):



FL Project playing:



As you can see, I have measured quite a substantial difference in VST based performance for both DAWs. The synths used are the same, the routing is the same, the automation is the same and the effects used are the same. Noteworthy: Reason idles at a higher CPU usage than FL when it's playing the project.

All tests were conducted on the following system:
CPU: i7 8700K OC at 5 GHZ
RAM: 32 GB Crucial Ballistix Sport 2400 in Dual Channel
ASUS Prime Z370 Series Motherboard
Presonus Audiobox iTwo
Driver used was FL ASIO at 512 smp latency, triple buffer OFF in FL

In the few months that I've been using Reason and especially working with this benchmark (a lot of opening FL, closing Reason and vice versa) I've also noticed some other quirks that bog down my experience with the program- namely the mandatory VST scanning BEFORE it even opens up- I have thousands of plugins, many of them from 32-bit days that are being scanned and disregarded every time I open Reason. It sometimes takes over 5 minutes for me. My suggestion would be to have an option in the plugin manager to only trigger scans manually- Anyone who's installed a plugin knows there is a plugin Reason doesn't know about, and it should be no problem to go to the manage plugins window and hit scan. I'm sure I'm not the only one who would welcome that workflow change.
The GUI of many VST plugins seems to run at a lower frame rate than reason- The GUI will lag, but the change in sound is smooth, so I'm guessing it's mainly a graphical thing...

If you own both FL and Reason and want to seethe difference in performance for yourself, I'm going to provide links to both sessions here. Be advised I work closely with Image Line, and therefore the version I made the benchmark with has some alpha code in there and therefore doesn't open with 12.5.1 build 165. It's old alpha code though, so it will open with the latest beta version of FL.

Single Benchmark FL:
https://www.dropbox.com/s/jazkkpyowopv3 ... e.flp?dl=0
Single Benchmark Reason:
https://www.dropbox.com/s/t2xvuxieglb48 ... eason?dl=0
Multi Benchmark FL:
https://www.dropbox.com/s/2is38lk5stt2v ... i.flp?dl=0
Multi Benchmark Reason:
https://www.dropbox.com/s/tthp4oyjtcvk4 ... eason?dl=0

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

Hey, so i helped with making the original benchmark file (with some automations borrowed from this lovely vanilla benchmark), i wanted to make a semi-realistic FX chain that i would usually use in a project, and then obviously we duplicated it all over and over to simulate an extreme workload.

Here is the original chain:
Plugins that contain automated parameters marked with [A]

Master: FabFilter Pro-L + HOFA 4U Meter Fader + Voxengo SPAN

Synth Bass: Serum[A] + FabFilter Pro-Q 2[A] + Fabfilter Pro-R[A] + OTT + SPAN

Drums: Addictive Drums 2(on fairfax demo kit) + FabFilter Pro-L + HODA 4U Meter Fader

Pad/Chord synth: IL Harmor[A] + IL Vocodex + FabFilter Pro-Q 2[A] + ValhallaRoom + HOFA 4U Meter Fader

Arpeggio: Serum[A] + Fabfilter Pr-L +HOFA 4U Meter Fader

Keys: Addictive Keys(demo) + ValhallaRoom[A] + Fabfilter Pro-Q 2[A]


I wanted to make a close to real-use scenario for how i automate reverbs/EQs etc so applied automations to the necessary parameters etc to simulate a general workload.

I made sure SH1 made the FL replica as close as possible to the reason version by bouncing him automations etc and making sure everything was duplicated in the same way, as FL doesn't duplicate devices+automations as easy as reason does, but wanted to make sure we got an apples-to-apples comparison.

My specs for my rig are this:
i9 7900X CPU (OCed to 4.3 GHz all cores)
32GB DDR4 Gskill Trident-Z 3000MHz RAM in quad channel config
Asus Strix 1080Ti
Asus TUF X299 Mark 1 mobo
EVGA 850G2 PSU
Samsung M.2 Boot SSD
Focusrite Scarlett 2i4

I don't want this post to seem like a negative thing towards Reason & Propellerhead because i absolutely love Reason and them, and have worked with them a lot too. I just wanted to have a tool to directly measure performance against another established DAW, and before VST was implemented we had no real way of doing so, as you could not get a fair comparison.

That being said, the results are quite huge, and given that VST was just implemented in Reason its understandable that there is a lot of room for optimisation, but the amount of difference is quite staggering, especially on my i9 CPU.

Here is the album of results in full

Both DAWs were set to buffer sizes of 442 samples on my Focusrite Scarlett 2i4 ASIO

Reason basic benchmark filer at idle
Image

FL basic benchmark file at IDLE
Image

Reason basic benchmark file PLAYING
Image

FL basic benchmark file PLAYING
Image

As you can see, even on the basic file the difference is huge. In both the basic and complex file, FL manages to play the song at lower CPU than reason can idle it.

Here are my results for the complex:

Reason complex benchmark file at IDLE
Image

FL complex benchmark file at IDLE
Image

Reason complex benchmark file PLAYING (gets to 100% cpu around the 4th/5th duplicate)
Image

Reason complex benchmark file STOPPED PLAYING @ 48 secs (CPU load too high)
Image

FL complex benchmark file PLAYING
Image


So in conclusion, you can see that the FL version never gets above even 50% usage at the end of the file, whereas Reason will hit 100% cpu even at around the 4th to 5th duplicate midi/instances and at that point the project is basically unworkable, windows becomes insanely laggy and can not really click on anything (and this is on an i9 CPU with 20 hyperthreads).

Feel free to download the tests and run them yourself, if you have the VSTs! And if youre only missing a few, maybe even delete those respective vsts and run that instead, you only need a relative comparison anyway.

Also, if someone could let me know how to spoiler tag in a post it would be appreciated, id rather not take up 10 pages with imgur embeds haha.

User avatar
Loque
Moderator
Posts: 11170
Joined: 28 Dec 2015

18 Jan 2018

Well, not really big news. Reason is to slow compared to other DAWs. Feel free to ask PH about an improvement. Better to see that fixed in the next version instead a more and more content delivery.

The always anyoing scanning of VSTs also happens on my side. I contacted the support and they said, the night he issues with vst3, console only and 32bit VSTs. Well, not very helpful, i dont want to clean and check 1000 files if they meet there expectations. Again, feel free to contact support...
Reason12, Win10

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

Loque wrote:
18 Jan 2018
Well, not really big news. Reason is to slow compared to other DAWs. Feel free to ask PH about an improvement. Better to see that fixed in the next version instead a more and more content delivery.

The always anyoing scanning of VSTs also happens on my side. I contacted the support and they said, the night he issues with vst3, console only and 32bit VSTs. Well, not very helpful, i dont want to clean and check 1000 files if they meet there expectations. Again, feel free to contact support...
This post wasn't supposed to be 'big news', its supposed to be a direct and measurable comparison, to put things into perspective and provide us with a metric of which to measure performance against, especially for future versions of reason.

User avatar
Psuper
Posts: 524
Joined: 29 May 2016

18 Jan 2018

Heya Protostar, I appreciate the work you put into this - I know it took a ton of time and effort, and might translate better for other users who aren't as aware of Propellerheads history or anticipate where they're headed.

I know I wouldn't have bothered because Propellerhead all but ignores Reasons backend and options, and no matter what is said where doesn't seem to make a difference. But these sort of tests are difficult to pull off and to me it seemed you did a stellar job. I hope it resonates with Propellerhead.

You'll find a lot of people here don't wanna hear about these sort of glaring shortcomings, but many of us do because ultimately (if Props listen) it helps us all even if they don't realize or appreciate it.
Reason needs to DAW.viewtopic.php?f=6&t=7504985

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

Psuper wrote:
18 Jan 2018
Heya Protostar, I appreciate the work you put into this - I know it took a ton of time and effort, and might translate better for other users who aren't as aware of Propellerheads history or anticipate where they're headed.

I know I wouldn't have bothered because Propellerhead all but ignores Reasons backend and options, and no matter what is said where doesn't seem to make a difference. But these sort of tests are difficult to pull off and to me it seemed you did a stellar job. I hope it resonates with Propellerhead.

You'll find a lot of people here don't wanna hear about these sort of glaring shortcomings, but many of us do because ultimately (if Props listen) it helps us all even if they don't realize or appreciate it.
Don't forget SH1s fantastic work of constructing the FL file too :D without him this would have not been possible whatsoever really haha.

And yeah hopefully it sheds some light on the issues at hand, and i do have good contact with people at prophead so i may forward this to them personally, but like i said, VST implementation just came into reason, so of course its not going to be optimal just yet.
I don't want to discourage them by saying 'vst implementation is bad right now' etc because they did a fantastic job with even implementing it in the first place, but its good to know where the shortcomings lie so that they can be measured and action taken against them!

Przemyslaw
Posts: 81
Joined: 05 Jan 2018

18 Jan 2018

I didn't know that the difference in vst support is so big. I work only in Reason. Hopefully, this year the new version of Reason will be much more efficient.

User avatar
Loque
Moderator
Posts: 11170
Joined: 28 Dec 2015

18 Jan 2018

Protostar wrote:
18 Jan 2018
Loque wrote:
18 Jan 2018
Well, not really big news. Reason is to slow compared to other DAWs. Feel free to ask PH about an improvement. Better to see that fixed in the next version instead a more and more content delivery.

The always anyoing scanning of VSTs also happens on my side. I contacted the support and they said, the night he issues with vst3, console only and 32bit VSTs. Well, not very helpful, i dont want to clean and check 1000 files if they meet there expectations. Again, feel free to contact support...
This post wasn't supposed to be 'big news', its supposed to be a direct and measurable comparison, to put things into perspective and provide us with a metric of which to measure performance against, especially for future versions of reason.
What is your point?

Mine is, nobody here can make reason faster so you should communicate this to PH, that was my suggestion. I appreciate your work and PH should consider this.
Reason12, Win10

User avatar
SH1audio
Posts: 4
Joined: 18 Jan 2018

18 Jan 2018

Loque wrote:
18 Jan 2018
What is your point?

Mine is, nobody here can make reason faster so you should communicate this to PH, that was my suggestion. I appreciate your work and PH should consider this.
The point of this thread is to show tangible evidence of how much of a difference there is in reality vs what I initally perceived as a multi-DAW user. And to encourage others to measure for themselves. I do not expect anything in Reason to change based on this post alone. We could have just contacted Propellerheads with the data, but making it publicly available first is imperative, because it invites discourse among users. Which is the thing that ultimately drives progress.

(and frankly it also was too much work to not show it to anyone else IMO)

groggy1
Posts: 466
Joined: 10 Jun 2015

18 Jan 2018

Awesome data. Very much appreciated that you shared it.

I have a feeling that 10.5 will hopefully be a big “engine update”. I went through the same thing with sonar before it had an engine rewrite.

I’m hoping that the reason 10.0 was content only is because maybe they’re working hard on a rehaul, and can’t touch the 10.0 engine without retesting all vsts. Just speculation though. :)

Crossing fingers for 10.5 engine overhaul

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

groggy1 wrote:
18 Jan 2018
Awesome data. Very much appreciated that you shared it.

I have a feeling that 10.5 will hopefully be a big “engine update”. I went through the same thing with sonar before it had an engine rewrite.

I’m hoping that the reason 10.0 was content only is because maybe they’re working hard on a rehaul, and can’t touch the 10.0 engine without retesting all vsts. Just speculation though. :)

Crossing fingers for 10.5 engine overhaul
It seems that Reason has a somewhat 'tick tock' update process, tick is content updates and tock is usually improvements to DAW, so we can only hope!

User avatar
aeox
Competition Winner
Posts: 3222
Joined: 23 Feb 2017
Location: Oregon

18 Jan 2018

I quit FL for Reason last year in February.

Prior to VST support I really didn't notice DSP issues. Only after getting VST, things started to fall apart.

It's because all RE and VST are running at 64 samples regardless of what your DAW is set to, because of the nature of what Reason is with it's CV modulation capability. Not sure if it will ever perform on par with other more simple DAWs.

Who knows, maybe they've got some tricks up their sleeves!

User avatar
Oquasec
Posts: 2849
Joined: 05 Mar 2017

18 Jan 2018

They make a good point. So drop Reason for vst support and use a second daw for that, Dibs on Reaper then as vst host
Producer/Programmer.
Reason, FLS and Cubase NFR user.

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

Oquasec wrote:
18 Jan 2018
They make a good point. So drop Reason for vst support and use a second daw for that, Dibs on Reaper then as vst host
Not really what we are saying :)

User avatar
Oquasec
Posts: 2849
Joined: 05 Mar 2017

18 Jan 2018

If ya need a daw without the mandatory buffer thing that was because of (rumor has it) the Cv ports needing it to be like that (not sure if that's why)
I am not gonna bother complaining and just use a second damn daw, to do extra vst stuff without bouncing in place.
Producer/Programmer.
Reason, FLS and Cubase NFR user.

kitekrazy
Posts: 1036
Joined: 19 Jan 2015

18 Jan 2018

I'm more of an FL fan boy but I think you need to give Props some time in improving VST support. I went through this ride when Cakewalk was DX only and went to VST in Sonar 2 via a wrapper. A lot of VSTs would not work in Sonar back then. Eventually Reaper came out and had superior VST support. I use to test VSTs in Reaper before adding them. I'd bet if Logic was ported back to PC there would be all sorts of problems at first. I still like the proprietary environment of Reason.

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

18 Jan 2018

kitekrazy wrote:
18 Jan 2018
I'm more of an FL fan boy but I think you need to give Props some time in improving VST support. I went through this ride when Cakewalk was DX only and went to VST in Sonar 2 via a wrapper. A lot of VSTs would not work in Sonar back then. Eventually Reaper came out and had superior VST support. I use to test VSTs in Reaper before adding them. I'd bet if Logic was ported back to PC there would be all sorts of problems at first. I still like the proprietary environment of Reason.
Not sure if you didnt read my comment all the way through but i basically expressed this sentiment.

This isnt a hate thread, its a tool for us to be able to measure any sizable changes/improvements and to see in hard data what state its currently in.

mataya

19 Jan 2018

I would also give prop time to improve vst support, but for god sake don't make me pay every time you do that.
I'd be more then happy to sell my license, but then I loose my rack investment.
God, so many problems with this piece of software.

User avatar
EnochLight
Moderator
Posts: 8405
Joined: 17 Jan 2015
Location: Imladris

20 Jan 2018

I appreciate this thread and the time the OP's put into making the benchmark song file. I've experienced similar results with Studio One 3.5.x and Reason 10.x, so every bit of information helps. Point Props to this thread and maybe they can use the data.
Win 10 | Ableton Live 11 Suite |  Reason 12 | i7 3770k @ 3.5 Ghz | 16 GB RAM | RME Babyface Pro | Akai MPC Live 2 & Akai Force | Roland System 8, MX1, TB3 | Dreadbox Typhon | Korg Minilogue XD

User avatar
Protostar
Posts: 27
Joined: 02 Apr 2017

20 Jan 2018

EnochLight wrote:
20 Jan 2018
I appreciate this thread and the time the OP's put into making the benchmark song file. I've experienced similar results with Studio One 3.5.x and Reason 10.x, so every bit of information helps. Point Props to this thread and maybe they can use the data.
I already sent it directly to them, so hopefully they can make use of the benchmark files to help with optimisation :)

Post Reply
  • Information
  • Who is online

    Users browsing this forum: No registered users and 24 guests