Apple Silicon implications for Rack Extensions

[Billy+]

I've been looking at what's required for building native apps and one question that popped out was about optimising Rack extensions.

I'm not a developer but do have a limited understanding of development and programming and after watching how to create universal applications for AS I started seeing problems with the way rack extensions work and how potential difficult creating optimised version maybe without a new SDK and Build system.

Maybe I'm over thinking it or probably just misunderstanding it but it's an interesting topic and I would like to hear what actual developers are doing/thinking about potential improvements to their RE's might be achieved as well as any problems that currently can't be worked around.

Is the holdup to AS mainly due to rack extensions and 3rd parties or are you all just waiting for RS to create the fix within a new SDK?

[pongasoft]

I've been looking at what's required for building native apps and one question that popped out was about optimising Rack extensions.

I'm not a developer but do have a limited understanding of development and programming and after watching how to create universal applications for AS I started seeing problems with the way rack extensions work and how potential difficult creating optimised version maybe without a new SDK and Build system.

Maybe I'm over thinking it or probably just misunderstanding it but it's an interesting topic and I would like to hear what actual developers are doing/thinking about potential improvements to their RE's might be achieved as well as any problems that currently can't be worked around.

Is the holdup to AS mainly due to rack extensions and 3rd parties or are you all just waiting for RS to create the fix within a new SDK?

The way I see it is 2 fold:

1) on one hand because the REs that are uploaded by developers are bitcode (not native code), then it should not be terribly hard to make a native/M1 version of the plugin as it should mostly be a matter of compiling with the right toolchain/set of flags
2) the biggest issue I can think of is the fact that, unless I misunderstood how it works, if an application is native/M1 then it can obviously load native/M1 plugins (which should be easy to do for RE, see point #1) but it cannot load x86 plugins (or in other words, the Rosetta layer kicks in at the application level not on a per plugin basis). So it means that all VSTs that do not have a native/M1 version (or are not universal meaning they have both versions included) will simply not load... and my guess is because Reason only support VST2 which is a very old version, it will be like 99.9% of them...

Yan

[orthodox]

What does "AS" mean?

As for RE optimization, I think they are optimized down to native code. Developers upload the intermediate representation (LLVM), which RS later translates into native code. So I guess when the next SDK with M1 support is out, all the old REs can be recompiled for M1 without developer involvement. New bugs may emerge as a result, but the native code will be properly optimized for M1.

[LABONERECORDINGS]

What does "AS" mean?

Apple Silicon

[orthodox]

Apple Silicon

Thanks.

[Billy+]

Ah LLVM and bitcode

So any actual optimisation would have to be inside the engine (Reason) not the actual bitcode (RE's)

Will the bitcode actually change then?

[orthodox]

So any actual optimisation would have to be inside the engine (Reason) not the actual bitcode (RE's)

Will the bitcode actually change then?

Reason is not the engine in the sense of a virtual machine executing bitcode. REs are compiled to native code and optimized at RS and downloaded by customers already as optimized native code libs.
The LLVM bitcode will not change in the next SDK. A new version of the C compiler will be provided, to make it possible for developers to test their REs if they're developing on M1.

[avasopht]

Ah LLVM and bitcode

So any actual optimisation would have to be inside the engine (Reason) not the actual bitcode (RE's)

Will the bitcode actually change then?

In a nutshell, Apple Silicon makes no effective difference whatsoever (apart from targeting the M1 processor, of course).

How it works is this:

Apple's LLVM toolchain will take the RE's LLVM bytecode and compile it to Apple Silicon while performing Apple Silicon specific optimisations.

So of course the final "bitcode" (typically referred to as the executable) has to be different because it's targeting a completely different CPU.

[crimsonwarlock]

all the old REs can be recompiled for M1 without developer involvement.

For me, this is one of the massive advantages that Reason has over other platforms: the fact that if Reason supports newer infrastructure, all REs automatically get support for that new infrastructure. The way they envisioned this beforehand is a stroke of genius. Even REs that are 'abandoned' by their developer, will still run on future systems. Compare this with the VST plugin situation, where you are completely dependent on the plugin developer to support new systems and/or standards. Like now that everyone is forced (by Steinborg) to the VST3standard, and all older VST2 plugins that won't be updated will be useless in the not so distant future.

This is one of the main reasons I moved to Reason. It is also one of the reasons I'm willing to cut them some slack when it comes to implementing new features and supporting new infrastructure. They already solve a big headache I had with other DAWs and plugins.

[dvdrtldg]

all the old REs can be recompiled for M1 without developer involvement.

For me, this is one of the massive advantages that Reason has over other platforms: the fact that if Reason supports newer infrastructure, all REs automatically get support for that new infrastructure. The way they envisioned this beforehand is a stroke of genius. Even REs that are 'abandoned' by their developer, will still run on future systems. Compare this with the VST plugin situation, where you are completely dependent on the plugin developer to support new systems and/or standards. Like now that everyone is forced (by Steinborg) to the VST3standard, and all older VST2 plugins that won't be updated will be useless in the not so distant future.

This is one of the main reasons I moved to Reason. It is also one of the reasons I'm willing to cut them some slack when it comes to implementing new features and supporting new infrastructure. They already solve a big headache I had with other DAWs and plugins.

As the owner of a lot of VST plugins and a brand new M1 Mac, I endorse this opinion

[Enlightenspeed]

all the old REs can be recompiled for M1 without developer involvement.

Like now that everyone is forced (by Steinborg) to the VST3standard, and all older VST2 plugins that won't be updated will be useless in the not so distant future.

Just a little add-on to note that VST3 is in no better position than VST2 in this regard.

Of course, you can use the JUCE or CMake frameworks to make it easier, but not everyone does this, and it may not even be possible for all developers - specifically those porting legacy plugins to new platforms. There is no guarantee with these 'universal' toolchains that what works on Windows will work on Mac etc, and what you do with these platforms is build multiple OS specific versions.

The only way to be sure is to prohibit any reliance on OS libraries, which is precisely what Reason Studios did with the RE SDK, which is effectively C++ and Lua with all such dependencies removed.

[Enlightenspeed]

I've been looking at what's required for building native apps and one question that popped out was about optimising Rack extensions.

Hi,

Ok, first things first.
Rack Extensions are not native apps.

I'm not a developer but do have a limited understanding of development and programming and after watching how to create universal applications for AS I started seeing problems with the way rack extensions work and how potential difficult creating optimised version maybe without a new SDK and Build system.

...and again, there is no such thing as Universal Applications for 'AS' - it's either universal or it isn't, basically.
What problems did you foresee?
The compiler will most likely just be the standard package that works for M1, adapted to work on the build server.

Maybe I'm over thinking it or probably just misunderstanding it but it's an interesting topic and I would like to hear what actual developers are doing/thinking about potential improvements to their RE's might be achieved as well as any problems that currently can't be worked around.

The improvements wouldn't be done by a developer themselves, the optimizations for REs are carried out by the compiling build server. Developers write "raw" C++/Lua code at a local level, but then export a universal build to the RS servers by simply using a different command line argument - so the only difference to the dev is literally typing either this:

python build45.py local45 Testing

or this:
python build45.py universal45

The former is platform specific to enable rapid prototyping to be completely localised, thus eliminating the potentially massive time loss waiting in build queues when you just wanted to change the shade of a single rectangle in a custom display, or alter a default setting in the motherboard etc.

Is the holdup to AS mainly due to rack extensions and 3rd parties

Unlikely. REs have faaaaaaaaar less to worry about in this scenario, and it's more likely that catering for VSTs would be a much bigger issue.

are you all just waiting for RS to create the fix within a new SDK?

Effectively, yes, but the difference to the developer will likely be rather transparent, as explained above. In fact, hi-res has more impact now on the dev than the M1 change will likely generate.

Hope this helps,
Brian

[crimsonwarlock]

Like now that everyone is forced (by Steinborg) to the VST3standard, and all older VST2 plugins that won't be updated will be useless in the not so distant future.

Just a little add-on to note that VST3 is in no better position than VST2 in this regard.

That seemed obvious to me, so I didn't state that the way VST is implemented means that this problem will occur again with future updates to the VST standard. Again, that should be obvious.

[Enlightenspeed]

Just a little add-on to note that VST3 is in no better position than VST2 in this regard.

That seemed obvious to me, so I didn't state that the way VST is implemented means that this problem will occur again with future updates to the VST standard. Again, that should be obvious.

Indeed, and I believe you

The point is intended for all readers, as this had not been made clear.

[Billy+]

Oh ok, so any and all improvements in performance would have to be implemented by the SDK and not the developers.

[Enlightenspeed]

Oh ok, so any and all improvements in performance would have to be implemented by the SDK and not the developers.

Not exactly. The compiler tool chain needs to be extended but the SDK would probably stay exactly the same.

[crimsonwarlock]

The point is intended for all readers, as this had not been made clear.

Fair enough