re-cmake open source release

[pongasoft]

I am re-posting here since it is the appropriate forum to do so and you might have missed it in the general one.

Hi guys

I am very excited to announce the open source release of re-cmake, the CMake based build framework for building Rack Extensions. Here are the main features:

• native support in IDEs handling CMake directly, like CLion or Visual Studio Code (with the CMake extension)
• project generation for most of the generators supported by CMake, like XCode for macOS, Visual Studio for Windows, Unix Makefiles, and many more
• completely isolated build (no polluting the source tree)
• much faster builds (due to proper dependency management, no need to recompile everything all the time)
• proper integration with IDEs for debugging (for example starting Recon from CLion in debug mode and putting a breakpoint in the source tree)
• native toolchain builds allowing for non sandboxed builds for easy development (use of std::string, std::cout, etc...)
• easy to use script (re.sh for macOS, re.bat for Windows) to invoke all the various phases of the build so no need to remember long and complicated syntax

In addition to the framework, it comes with a "blank" plugin (does nothing but is fully wired and runs inside Recon and on Reason servers), which serves dual purpose:

1) documentation for the framework
2) a project that can easily be cloned to start your own rack extension: it contains all the boilerplate code and file structure and because it uses the CMake build framework, you only need to change the info.lua file with your own names (and CMakeLists.txt for the project name and this is where you add your own graphics and source code) and you are good to go..

Here is a screenshot showing a few things:

• The project loaded in CLion with all the targets available
• The source code of the "main" method (does nothing / empty)
• The result of the native-install target run in CLion which has compiled and installed the plugin in its location so that Recon can find it
• A shell showing the command that can be run on the command line to do the same outside the IDE
• The plugin loaded in Recon

file.png (507.24 KiB) Viewed 30750 times

Finally I wanted to thank David Antliff from Pitchblende who provided much appreciated help and feedback during the development process.

Feel free to contact me personally or use the ticket/issue on github for feedback, bugs, etc...

Yan

[akeia]

Yan, your work is wonderful. I got the example to compile and show up in Recon.

I have all kinds of ideas for plugins or helpers for my music. But before I invest any work I have a big question: How to use my own plugins in Reason?

I mean it is fine to debug and test them in Recon but there is no way to use them in Reason?!?

I am used to the way Apple does it. You can use your own apps on your own phone without commercially publishing them. Without such a scheme I think opening up the Reason development is useless for non-commercial developers.

Is my observation correct that there is no way to use your own plugins in Reason(without commercially publishing them)? Do you think it makes any sense to lobby Reason Studios about this issue.

[Murf]

Yan, your work is wonderful. I got the example to compile and show up in Recon.

I have all kinds of ideas for plugins or helpers for my music. But before I invest any work I have a big question: How to use my own plugins in Reason?

I mean it is fine to debug and test them in Recon but there is no way to use them in Reason?!?

I am used to the way Apple does it. You can use your own apps on your own phone without commercially publishing them. Without such a scheme I think opening up the Reason development is useless for non-commercial developers.

Is my observation correct that there is no way to use your own plugins in Reason(without commercially publishing them)? Do you think it makes any sense to lobby Reason Studios about this issue.

Just upload them to the build servers and give yourself a beta of it

[akeia]

Just upload them to the build servers and give yourself a beta of it

I can not upload to the build servers unless I give some "company information". As an amateur developer I have no company. And registering one here in Europe opens up all kind of bureaucratic nightmares and has complicated and possibly expensive tax implications. That is a bit much for building my own plugins.

[pongasoft]

Just upload them to the build servers and give yourself a beta of it

I can not upload to the build servers unless I give some "company information". As an amateur developer I have no company. And registering one here in Europe opens up all kind of bureaucratic nightmares and has complicated and possibly expensive tax implications. That is a bit much for building my own plugins.

My understanding, is that you only need a company if you are going to charge for your plugins. I did create a company for pongasoft to publish my free RE (which costs me money...) but with the way things have evolved I believe you don't need one anymore for free stuff. I would contact Reason Studios directly.

It is true that you can publish your REs in beta, but they will expire so you will have to republish them over and over which is not great. Another solution is to release them for free (they don't have to be open source) even if they seem to be very specific to your needs.

Yan

[akeia]

... but with the way things have evolved I believe you don't need one anymore for free stuff. I would contact Reason Studios directly.

I will contact Reason Studios about the issue. My impression is that the information in the developer area of the website is a little neglected and outdated. I hope they do care about non-commercial development and that there will be an easy way to do that.

[Billy+]

I see the OS X version tested was 10.14.6 I'm running El Capitan 10.11.6 will the framework work for me?

I assume the reason recon sdk stuff will as I register some time ago, but this framework really has me interested in try some stuff

[Murf]

I see the OS X version tested was 10.14.6 I'm running El Capitan 10.11.6 will the framework work for me?

I assume the reason recon sdk stuff will as I register some time ago, but this framework really has me interested in try some stuff

Not sure but just give it a try!

[Murf]

Just upload them to the build servers and give yourself a beta of it

I can not upload to the build servers unless I give some "company information". As an amateur developer I have no company. And registering one here in Europe opens up all kind of bureaucratic nightmares and has complicated and possibly expensive tax implications. That is a bit much for building my own plugins.

Do you have the option in your Country of a "Business" in your personal name? In Australia we have them, it is very cheap and Reason Studios accepted it for my account.
(Here it is called "ABN")

Murf.

[Billy+]

I'm not sure but would be more interested in free / personal RE's

[pongasoft]

I just released a minor update to re-cmake (1.0.1):

* When providing multiple commands to the script, the script now properly fails on the failed command and terminates with the error code of the failing process
* Added -b option to the script to add a banner between multiple commands to make the output easier to follow

Yan

[Murf]

I just released a minor update to re-cmake (1.0.1):

* When providing multiple commands to the script, the script now properly fails on the failed command and terminates with the error code of the failing process
* Added -b option to the script to add a banner between multiple commands to make the output easier to follow

Yan

Thanks Yan!

[Oquasec]

If that actually works that be neat. The only question have is how did you create a u45 file successfully and submitted it?

[pongasoft]

If that actually works that be neat. The only question have is how did you create a u45 file successfully and submitted it?

You use the "universal45" command:

Code: Select all

> ./re.sh universal45
Scanning dependencies of target jbox-u45-build
Product ID: com.pongasoft.BlankPlugin
RackExtension name: BlankPlugin
Using LLVM at path: /Users/Shared/ReasonStudios/JukeboxSDK_4.2.0/SDK/Tools/LLVM
...
...
Creating archive Output/Universal45/BlankPlugin.u45
Build finished
Generated: /Volumes/Vault/deployment/build/re-blank-plugin/Debug/jbox/Output/Universal45/BlankPlugin.u45
[100%] Built target jbox-u45-build

Then you upload the .u45 created to the servers.. nothing else to do...

Yan

[pongasoft]

I just pushed a new release (1.1.0) with the following release notes:

* Added preview command (resp. common-preview build target) which runs the RE2DPreview tool provided with the SDK. This tool generates a 2D preview of the device front, back, folded front and folded back (generated at full 5x resolution (3770x345*u)). This can be useful to generate images required for the shop (vs running Reason and capturing a low resolution image).

Here is an example for CVA-7 front panel (note that I had to reduce the size by 50% to be able to upload to the forum...):

Snapshot_Panel_Front.png (745.37 KiB) Viewed 30363 times

Yan

[meowsqueak]

I just pushed a new release (1.1.0)

Excellent, thank you for this addition. I am very happy with your system and I am using it a lot.

[pongasoft]

I have been (successfully) working on making sure that Jamba runs and works properly on Apple M1 chipset. I will be releasing the changes in the next few days (build/CMake changes only, no code changes)

Once I am done, I will make sure that re-cmake works as well on it (which means forcing x86_64 compilation since RE SDK / Recon is x86_64 for the native build). I will also update the RE Quick Start tool to account for the changes.

I will post an update as soon as I am done. Thank you for your patience.

Thanks
Yan

[Reasonistas]

I have been (successfully) working on making sure that Jamba runs and works properly on Apple M1 chipset. I will be releasing the changes in the next few days (build/CMake changes only, no code changes)

Once I am done, I will make sure that re-cmake works as well on it (which means forcing x86_64 compilation since RE SDK / Recon is x86_64 for the native build). I will also update the RE Quick Start tool to account for the changes.

I will post an update as soon as I am done. Thank you for your patience.

Thanks
Yan

This is an excellent update. Thanks, Yan!

[pongasoft]

I have released re-cmake v1.2.0 which now works with the new Apple chipset https://github.com/pongasoft/re-cmake

* Introduced `cmake/RECMakeOptions.cmake` so that it can be invoked **prior** to `project()`.
* Builds properly on Apple chipset (by forcing the build in `x86_64` architecture)

This change is optional but if you want your project to adopt the new structure, you can simply generate a new blank plugin to see the changes (only affects `CMakeLists.txt`) or check the re-blank-plugin project.


In action:

Screen Shot 2021-01-05 at 11.27.35.png (488.21 KiB) Viewed 29877 times

[Billy+]

I have released re-cmake v1.2.0 which now works with the new Apple chipset https://github.com/pongasoft/re-cmake

* Introduced `cmake/RECMakeOptions.cmake` so that it can be invoked **prior** to `project()`.
* Builds properly on Apple chipset (by forcing the build in `x86_64` architecture)

This change is optional but if you want your project to adopt the new structure, you can simply generate a new blank plugin to see the changes (only affects `CMakeLists.txt`) or check the re-blank-plugin project.

You are on fire

[pongasoft]

I pushed a new version of re-cmake 1.3.1 which now, (optionally) lets you add unit tests for your REs and add a simple command to run them:

Code: Select all

> ./re.sh test                                                                                      
[ 14%] Built target gtest
[ 42%] Built target z-re-sdk-lib
[ 71%] Built target native-test-lib
[ 85%] Built target gtest_main
[100%] Built target native-build-test
Running tests using /Volumes/Vault/deployment/build/re-blank-plugin/Debug/native-build-test
Test project /Volumes/Vault/deployment/build/re-blank-plugin/Debug
    Start 1: Device.SampleRate
1/1 Test #1: Device.SampleRate ................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.01 sec
[100%] Built target native-run-test

• it uses Google Test
• re-quickstart has been updated accordingly so if you generate a brand new plugin, it will have a very basic unit test as a demonstration on how to write one.
• re-blank-plugin has a similar example:

  Code: Select all

  #include <logging.h>
  #include <gtest/gtest.h>
  #include <Device.h>
  
  // Device - SampleRate
  TEST(Device, SampleRate)
  {
    DLOG_F(INFO, "Demonstrating test capability");
    Device device{44100};
    ASSERT_EQ(44100, device.getSampleRate());
  }

Note that, at this time, it is only designed for "pure" unit tests which don't involve talking to the RE APIs (like JBox_LoadMOMProperty) since these calls are handled by the DAW. That being said, I have successfully been able to mock (some of) these calls and I will release (part of re-common) when it is more stable...

[meowsqueak]

Nice - this will be useful, thank you. Is it hooking into the CTest mechanism (i.e. gtest_discover_tests), or is there a better/newer way to do it now?

[pongasoft]

Nice - this will be useful, thank you. Is it hooking into the CTest mechanism (i.e. gtest_discover_tests), or is there a better/newer way to do it now?

I do not know what gtest_discover_tests is. This is how I did it . You simply pass the list of test source files (you can use whatever method you want to discover this list including GLOB pattern matching). If that doesn't suit your needs, you can write your own cmake/RECMakeAddTest.cmake file in your own project and it will be used instead.

[meowsqueak]

I do not know what gtest_discover_tests is.

Ah, ok, in a nutshell, CMake has a subsystem called CTest that can manage and execute Unit Tests (it's agnostic to Google Test, CUnit, etc). Usually this requires the developer to explicitly add each test to the CMakeLists file which can be onerous and error-prone. Sometimes you also end up with myriad executables, one per test, sometimes one binary, sometimes something in-between, but CTest abstracts most of that away for you.

gtest_discover_tests is a way to allow CTest to discover individual tests from a single binary created with Google Test (I imagine it can work for multiple binaries too). CTest can run the binary in a special way and get a list of all the tests within the binary, then CTest can run each test individually or filtered into groups, or however you want, without Google Test getting in the way.

It looks like your system does most of this anyway - it creates a single binary linked with Google Test's main (or your own) that you can run directly to run all the tests, is that right? So you still get all your Google Tests run together, which is what most people want anyway. This takes CTest out of the workflow but that's not a problem for most people either - CTest is mostly useful for more advanced usage like tracking CI and generating reports, it's not an essential tool.

[pongasoft]

I do not know what gtest_discover_tests is.

Ah, ok, in a nutshell, CMake has a subsystem called CTest that can manage and execute Unit Tests (it's agnostic to Google Test, CUnit, etc). Usually this requires the developer to explicitly add each test to the CMakeLists file which can be onerous and error-prone. Sometimes you also end up with myriad executables, one per test, sometimes one binary, sometimes something in-between, but CTest abstracts most of that away for you.

gtest_discover_tests is a way to allow CTest to discover individual tests from a single binary created with Google Test (I imagine it can work for multiple binaries too). CTest can run the binary in a special way and get a list of all the tests within the binary, then CTest can run each test individually or filtered into groups, or however you want, without Google Test getting in the way.

It looks like your system does most of this anyway - it creates a single binary linked with Google Test's main (or your own) that you can run directly to run all the tests, is that right? So you still get all your Google Tests run together, which is what most people want anyway. This takes CTest out of the workflow but that's not a problem for most people either - CTest is mostly useful for more advanced usage like tracking CI and generating reports, it's not an essential tool.

I see. It is easy to change to use this and it seems to work so I will change it. It seems that the big difference is that CMake does not have to regenerate the project every time you touch something which seems a lot better!