Benchmark Test between 10.2 and the upcoming 10.3

[Heigen5]

Ok, I'm not an expert about Macs, so thanks for clearing it up. Anyway, anyone wants to guess why Marco's Mac is not taking the advantage of the hyper-treading then?

There's no such thing as "taking advantage of hyperthreading" for audio processing - it's either the same performance (or perhaps maaaaarginallly better), or much worse:
viewtopic.php?f=4&t=7505905&p=421244

But if I have it on over the preferences, I receive about 30 secs more playback time, so it's not like it's an automated default task - you do need to have it on, so that's why it means "taking advantages of it".

[EnochLight]

There's no such thing as "taking advantage of hyperthreading" for audio processing - it's either the same performance (or perhaps maaaaarginallly better), or much worse:
viewtopic.php?f=4&t=7505905&p=421244

But if I have it on over the preferences, I receive about 30 secs more playback time, so it's not like it's an automated default task - you do need to have it on, so that's why it means "taking advantages of it".

It needs to be clarified (once again) that YMMV. Some people's performance will benefit with HT on, and some people will suffer. It also seems to vary depending on the project as well as the amount of devices in a particular device chain, per channel. There's no hard fast rule, it would appear.

[Heigen5]

But if I have it on over the preferences, I receive about 30 secs more playback time, so it's not like it's an automated default task - you do need to have it on, so that's why it means "taking advantages of it".

It needs to be clarified (once again) that YMMV. Some people's performance will benefit with HT on, and some people will suffer. It also seems to vary depending on the project as well as the amount of devices in a particular device chain, per channel. There's no hard fast rule, it would appear.

Ok, but we both (Marco and me) run the same Reason file to test it.

[ScuzzyEye]

HT is supposed to use unused CPU cycles as a "ghost"/virtual CPU core, but if you have a DSP channel that is very CPU heavy and must be calculated in a single thread then that might be a bottle neck. So, I can see how results seem inconsistent if that is the case. I'm not 100% sure but the HT performance is probably dependent on the actual song/RE/VST setup and if it can be split into more CPU threads or not.

You're on the right track. Hyperthreading helps to take advantage of running code that doesn't efficiently keep the CPU busy. Like code that doens't use a lot of SIMD (MMX, SSE, AVX, etc) streaming instructions. If there are two threads sharing the same core, and they don't alone keep the execution busy, it might be possible that re-ordering their instructions together will be more efficient. But if one of the threads is highly optimized, trying to mix it with another thread will only hurt both.

So it really is code dependent. Even closing Reason and reloading the same song file, can produce different results, as different plug-ins can end up on different cores. You can simply get a feel for how the plug-ins you usually use behave with or without Hyperthreading, and choose to leave it on or off depending on what better suits you.

And no it isn't possible for Reason/the OS/the CPU to be smart about how things are placed, because knowing how code is going to behave before you run it, is impossible. See the halting problem.

[antic604]

Fixed that for you.

Can you prove it does improve performance in other DAWs, though?

[EnochLight]

Fixed that for you.

Can you prove it does improve performance in other DAWs, though?

It was tongue and cheek. That said, all DAW's/plugins seem to handle it in their own way. I've had mixed bag results with Studio One (my only other DAW I can compare right now). How well does it work in Bitwig and Cubase for you?

[antic604]

It was tongue and cheek. That said, all DAW's/plugins seem to handle it in their own way. I've had mixed bag results with Studio One (my only other DAW I can compare right now). How well does it work in Bitwig and Cubase for you?

Neither gives option to toggle it on/off from the software and on Surface Pro 4 I'm locked out of BIOS, so can't tell

But I just ordered a new laptop, so finally I'll be able to test it (and quit complaining about high-DPI, because it's a 1080p pane )

[mcatalao]

Magic

Well, if anything this only confirms people complaining about Reason getting (much) slower after VSTs were added (even without using plugins, just native+REs) were right all along. Not that I would know, having started with 10...

Can't wait to get home and see it for myself - just got the acceptance e-mail today

That's not true. Introduction of VST's didn't change anything for projects without VST. Let's test things right before we say things from the top of our heads. Being a Beta Tester, we have the responsibility to say things with knowledge and some certainty.

My experience says R6 to 10.2 has the same performance on Native devices.
R8 to 10.2 has the same performance with the compatible Rack Extensions. You cannot test earlier sdk Re's on R7-.

Test this with the same settings and you will see this (R6 against R10 and R8 against R10). And don't forget to check the same CPU settings on reason (R8 didn't have Hyperthreading, so you shoud test r10 with HD disabled).

[mcatalao]

By the way, the benchmark file playd the whole file without any glitches in the sound.

Did you try to see if hyper-threading on/off made any difference?

Here's an interesting fact about hyperthreading (even on R10.2) in R10. I am able to run more synths with hyperthreading on than off, though processor use seemed bigger. And this happened because processor usage is stabler with HT than without. Reason "grabs" and controls more resources with HT on. It's like if reason had a bigger priority versus other applications and OS processes.

[EnochLight]

But I just ordered a new laptop, so finally I'll be able to test it (and quit complaining about high-DPI, because it's a 1080p pane )

Wait - don't ever stop complaining about the lack of high-DPI/4K support in Reason, because I need allies in my quest to get Props to wake up and change it!

[antic604]

Well, if anything this only confirms people complaining about Reason getting (much) slower after VSTs were added (even without using plugins, just native+REs) were right all along. Not that I would know, having started with 10...

That's not true. Introduction of VST's didn't change anything for projects without VST. Let's test things right before we say things from the top of our heads. Being a Beta Tester, we have the responsibility to say things with knowledge and some certainty.

My experience says R6 to 10.2 has the same performance on Native devices.

Sorry, just referencing what many had said here & elsewhere - there were even comments about "10.3 feeling like Reason 7/8 again" and not just from one person. I started with 10 so can't compare

But you're right - I should not repeat things I can't back up with my own experience.

[antic604]

Wait - don't ever stop complaining about the lack of high-DPI/4K support in Reason, because I need allies in my quest to get Props to wake up and change it!

Well, I'll still want full GPU acceleration, resizeable GUI (like Bitwig or Live) and vertically aligned Mixer and Rack (especially since I'm switching from 3:2 screen ratio to 16:9).

BTW, I've ordered this one. A friend got one 3 months ago (a lower model) and he can't seem to stop singing praises and he's coming from 15'' Macbook Pro!

[mcatalao]

Sorry, just referencing what many had said here & elsewhere - there were even comments about "10.3 feeling like Reason 7/8 again" and not just from one person. I started with 10 so can't compare

But you're right - I should not repeat things I can't back up with my own experience.

Well, i stand before what i say because i am a user since R3 and i have licenses for all versions since that.
From this point, someone with the same license level can go and download older versions to test it. So i tested it for R6, R8 and R10. And my tests showed me invariably that projects with native devices had marginally the same load across all tested versions.

For this i used an old Reason 6 project that i ran on R6 and R10. I did the same with a more "synth based" project with a lot of thors. Same result, project loaded marginally the same CPU same DSP bars on R6 and R10. These were old projects created with an old Q6600 box, i remember they loaded my CPU up to 60% and on my current machine they load more or less 10-20%. But the results are undeniable. Native device based projects with 20+ tracks loading something in between 10 to 20% CPU with marginally the same performance. Across R6 and R10.2.

I then created a R8 project with the Rack extensions that R8 allowed me to load. R8 worked with SDK 2.x and a lot of my Rack Extensions are already at a greater SDK level, so a lot of them do not load. You can't test R8 with Vecto but you can have a project with Antidotes, Parsec and ReSpire. So i created a project with about 15 synths. This project loaded about 30% CPU and 2-3 bars on Reason 8, and exactly the same number of bars in R10.2, and marginally the same CPU load (at some points even a little better). Anyway, this is a more real life project where i have a lot Re synths so it's normal that this starts at 30-50% on 10.2. And again, backing my own claims, i doubled the tracks, and R8 couldn't handle the doubled synths. R10 without hyperthreading also didn't handle it but with hyperthreding on it runs the project with no glitches. Since this findings, i always use Hyperthreading on despite it seems there more CPU load but the system runs a lot stabler.

An important reference is that these testings apply to my setup and i work with PC's. Maybe a lot of people complaining use Mac's or they have different spec'ed cpu's, ram, OS, etc. However have a hard time to believe PC based users have such different experiences.

[Heigen5]

Why do you people benchmark 8 and 10.2 - as it's the 10.3 that changed things in the CPU-wise?

[Loque]

Why do you people benchmark 8 and 10.2 - as it's the 10.3 that changed things in the CPU-wise?

It is good for comparison. Appreciate what the ppl here are doing! Keep up the great work!

[antic604]

Why do you people benchmark 8 and 10.2 - as it's the 10.3 that changed things in the CPU-wise?

Read carefully. I reiterated that other posters said 10.3 is like 7/8 again, which suggests 10.2 (and 9.5) were slower than those old versions. To be fair, that complaint also appeared multiple times before 10.3

The other poster just says that in his experience 10 wasn't any slower than 8 or even 6 when using native devices. And 10.3 would likely make it faster. That's it

[tronam]

Not bad! The complex benchmark used to instantly fail before even hitting play no matter what buffer setting I used. Now on 10.3 I can make it to 15 bars before hearing any pops or clicks.

[BananaSkins]

Not bad! The complex benchmark used to instantly fail before even hitting play no matter what buffer setting I used. Now on 10.3 I can make it to 15 bars before hearing any pops or clicks.

What buffer setting?

[tronam]

Not bad! The complex benchmark used to instantly fail before even hitting play no matter what buffer setting I used. Now on 10.3 I can make it to 15 bars before hearing any pops or clicks.

What buffer setting?

512 buffer, which tends to be the sweet spot for me most of the time since I rarely pound on drum pads.

[mcatalao]

Guys,

Here some thoughts ( i was going to call them myth busters but some might think offensive) from stuff that i have noticed empirically:

1 - At the risk of sounding like a broken record, there's no difference in performance between R6, R8 and R10.2, as long as you guarantee the conditions that apply for each Version while comparing are equal. So for example if you compare a R6 project you don't have Rack extensions, and there wasn't any kind of hyper threading so, you must create a project that respects that (easy way, download R6, create a project there, check DSP/CPU, load in 10.2 disable HT, compare).

Same for R8, create a project there (you have to pass a bunch of Re loading errors because a lot of those were reviewed to the newest SDK), check the project DSP/CPU load, open the project it in 10.2, disable HT and check load, compare.

If you do the tests right, and if you put away any kind of difference (heavier Re's or VST's) performance will be the same. I've done this, i even have a video with this tested, but i don't think it is worth to post it when people are focusing on R10.3 now.

2 - HT can improve some systems (mine is the case). However i can't be sure if whats' happening with my system is a result of HT directly or a shift in the affinity of the cpus, giving reason an extra real CPU.
You see without HT audio rendering, Reason creates threads on CPU 0, 2, 4, and the last CPU is always under 10%. When i set Hyper threading audio rendering on, Reason creates threads to all CPU's except for CPU7. So with HT on, Reason loads threads on cpu's 0,1,2,3,4,5,6 (7 cpu's of eight), but considering the pair number cpu's are real cores (because initially the threads are created on 0, 2, 4), reason gets another thread at CPU 6 which is my case a real core (this is just a theory right?). The fact is, and again only empirically speaking and (again) might not even be a direct result of it, IN MY SYSTEM Hyper Threding Audio Rendering enabled helps a lot, as you can see in the video:

3 - I've never been very fond of this crazy benchmark tests, but they are important for comparing rigs, and see Reason performance evolution over the time and different versions. That being said, i tested the Complex Benchmark and while in Reason 10.2, my system could barely run 1 bar, after R10.3 with HT Audio Rendering and the Render audio using audio card buffer on, it plays up to 57 secs (oh and without HT it only run 35 secs).

That's it. Now you can kill me!

[mcatalao]

PS.: i have a number four:

4 - You don't believe me, assume it's true or test it, but since in my system 1 and 2 and 3 are true, the fun part is Reason runs now more efficiently than any other version above 5.

[Retrotrollet]

Multicore in 10.3 seems to work no pops or clicks
12 real cores HT off
Asio off for screen recording.

[Retrotrollet]

Benchtest Complex RE.
HT off, 12 real cores.
ASIO OFF ( for screen recording )
Pretty sweet multicore usage

[tronam]

I just got my new 8-core i9 9900k system yesterday and it's been interesting to see the before/after results with the "complex" benchmark. Even with a significantly more powerful processor I can still only make it to bar 19 before it starts popping and skipping in v10.2 mode. In v10.3 mode (512 samples / HT OFF) I can make it all the way to the end at bar 45 without a single hiccup. The difference is dramatic.