[re-mock] A testing framework for REs (beta)

This forum is for developers of Rack Extensions to discuss the RE SDK, share code, and offer tips to other developers.
Post Reply
User avatar
pongasoft
RE Developer
Posts: 478
Joined: 21 Apr 2016
Location: Las Vegas
Contact:
Contact pongasoft

13 Sep 2021

Hello

I wanted to announce the release (open source / free) of re-mock, a framework to unit (or black box) test rack extensions.

The project is on github: re-mock.

Taste of the framework:

Code: Select all

Example of unit test (using GoogleTest):

// Creates a config by reading motherboard_def.lua and realtime_controller.lua
auto c = DeviceConfig<Device>::fromJBoxExport(RE_CMAKE_MOTHERBOARD_DEF_LUA, RE_CMAKE_REALTIME_CONTROLLER_LUA);

// Creates a tester for the device (a studio_fx device)
auto tester = StudioEffectTester<Device>(c);

// Wire the sockets
tester.wireMainIn("audioInLeft", "audioInRight");
tester.wireMainOut("audioOutLeft", "audioOutRight");

// Change properties (simulate user action)
tester.device().setNum("/custom_properties/gain", 0.7);

// Call "JBox_Export_RenderRealtime" of all registered devices
// Populate a stereo buffer (2x64 samples) with the value 0.5 (left) and 0.6 (right) 
// made available to the device under test on the [in] sockets previously wired ("audioInLeft" / "audioInRight")
// Read the [out] sockets after processing and return it (buffer)
auto buffer = tester.nextFrame(MockAudioDevice::buffer(0.5, 0.6));

// Check that the resulting buffer is what we expected (here we assume that it is an effect with a gain knob
// set to 0 dB change => output == input).
ASSERT_EQ(MockAudioDevice::buffer(0.5, 0.6), buffer);

// Make sure that properties were changed accordingly
ASSERT_TRUE(tester.device().getBool("/custom_properties/sound_on_led");
The framework essentially implements a "mock" version of the reason rack including:
  • Support for the Jukebox api (JBox_... api in Jukebox.h)
  • Rack / Motherboard concept with properties
  • Lua parsing (motherboard_def.lua / realtime_controller.lua) to load the actual configuration of the device
  • Lua code execution to instantiate the device (C++ -> lua -> C++)
  • Very convenient APIS to access the device itself or the properties of the motherboard (get/set)
  • "Testers" concept to facilitate testing of the device
  • And much much more...
This is currently beta as there are still a few missing apis mostly because I do not have a good example for them. I would definitely appreciate any feedback and if you are interested in helping to add the missing pieces (even if it is just to provide advice and/or examples of what you need), do not hesitate to reach out.

Yan
Top
User avatar
pongasoft
RE Developer
Posts: 478
Joined: 21 Apr 2016
Location: Las Vegas
Contact:
Contact pongasoft

13 Sep 2021

For a more complete example, you can check the test added for A/B Switch on github.

Here is the code:

Code: Select all

#include <gtest/gtest.h>
#include <Device.h>
#include <re/mock/DeviceTesters.h>
#include <re_cmake_build.h>

using namespace re::mock;

/**
 * Represents the state of the device from an external point of view */
struct State
{
  static constexpr bool A = false;
  static constexpr bool B = true;

  bool soundOn{false};
  bool audioLEDA{false};
  bool cvLEDA{false};
  bool audioLEDB{false};
  bool cvLEDB{false};
  bool audioSwitch{A};
  bool cvSwitch{A};
  bool xFade{false};

  bool operator==(State const &rhs) const;

  bool operator!=(State const &rhs) const;

  std::string to_string() const
  {
    return fmt::printf("soundOn=%d, audioLEDA=%d, cvLEDA=%d, audioLEDB=%d, cvLEDB=%d, audioSwitch=%d, cvSwitch=%d, xFade=%d",
                       soundOn, audioLEDA, cvLEDA, audioLEDB, cvLEDB, audioSwitch, cvSwitch, xFade);
  }

  // Create a free inline friend function.
  friend std::ostream& operator<<(std::ostream& os, const State& s) {
    return os << s.to_string();  // whatever needed to print bar to os
  }

  static State from(HelperTester<Device> &iTester)
  {
    auto re = iTester.device();
    
    return {
      .soundOn = re.getBool("/custom_properties/prop_soundOn"),
      .audioLEDA = re.getBool("/custom_properties/prop_audio_ledA"),
      .cvLEDA = re.getBool("/custom_properties/prop_cv_ledA"),
      .audioLEDB = re.getBool("/custom_properties/prop_audio_ledB"),
      .cvLEDB = re.getBool("/custom_properties/prop_cv_ledB"),
      .audioSwitch = re.getBool("/custom_properties/prop_audio_switch"),
      .cvSwitch = re.getBool("/custom_properties/prop_cv_switch"),
      .xFade = re.getBool("/custom_properties/prop_xfade_switch")
    };
  }
};

bool State::operator==(State const &rhs) const
{
  return soundOn == rhs.soundOn &&
         audioLEDA == rhs.audioLEDA &&
         cvLEDA == rhs.cvLEDA &&
         audioLEDB == rhs.audioLEDB &&
         cvLEDB == rhs.cvLEDB &&
         audioSwitch == rhs.audioSwitch &&
         cvSwitch == rhs.cvSwitch &&
         xFade == rhs.xFade;
}

bool State::operator!=(State const &rhs) const
{
  return !(rhs == *this);
}

MockAudioDevice::buffer_type xfade(MockAudioDevice::buffer_type const &iFromBuffer,
                                   MockAudioDevice::buffer_type const &iToBuffer)
{
  MockAudioDevice::buffer_type res{};
  for(int i = 0; i < MockAudioDevice::NUM_SAMPLES_PER_FRAME; i++)
  {
    double f = static_cast<double>(i) / (MockAudioDevice::NUM_SAMPLES_PER_FRAME - 1.0);
    res[i] = (iToBuffer[i] * f) + (iFromBuffer[i] * (1.0 - f));
  }
  return res;
}

MockAudioDevice::StereoBuffer xfade(MockAudioDevice::StereoBuffer const &iFromBuffer,
                                   MockAudioDevice::StereoBuffer const &iToBuffer)
{
  return { .fLeft = xfade(iFromBuffer.fLeft, iToBuffer.fLeft), .fRight = xfade(iFromBuffer.fRight, iToBuffer.fRight) };
}

// Device - SampleRate
TEST(Device, SampleRate)
{
  auto c = DeviceConfig<Device>::fromJBoxExport(RE_CMAKE_MOTHERBOARD_DEF_LUA, RE_CMAKE_REALTIME_CONTROLLER_LUA);
  auto tester = HelperTester<Device>(c);

  ASSERT_EQ(44100, tester.device()->getSampleRate());

  State s{};
  ASSERT_EQ(s, State::from(tester));
}

// Device - AudioSwitch
TEST(Device, AudioSwitch)
{
  auto c = DeviceConfig<Device>::fromJBoxExport(RE_CMAKE_MOTHERBOARD_DEF_LUA, RE_CMAKE_REALTIME_CONTROLLER_LUA);
  auto tester = HelperTester<Device>(c);

  auto srcA = tester.wireNewAudioSrc();
  auto srcB = tester.wireNewAudioSrc();
  auto dst = tester.wireNewAudioDst("audioOutputLeft", "audioOutputRight");
  auto srcCV = tester.wireNewCVSrc();

  State s{};

  ASSERT_EQ(s, State::from(tester));

  /*******************************/////////////******************************/
  /*                             < FIRST FRAME >                            */
  /*******************************/////////////******************************/
  tester.nextFrame();

  ////////// Checks //////////

  // after the first frame, the switch is now in its default state where A is selected for audio and cv (set by RT)
  // no input connected => audio should stay at 0
  s.audioLEDA = true;
  s.cvLEDA = true;
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(0, 0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - wiring srcA -> abSwitch, srcB -> abSwitch //////////

  tester.wire(srcA, "audioInputLeftA", "audioInputRightA");
  tester.wire(srcB, "audioInputLeftB", "audioInputRightB");

  tester.nextFrame();

  ////////// Checks - input is still 0/0 //////////

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(0, 0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - setting A to 1/2 and B to 3/4 //////////

  srcA->fBuffer.fill(1.0, 2.0);
  srcB->fBuffer.fill(3.0, 4.0);

  tester.nextFrame();

  ////////// Checks //////////

  // since A is selected, we should get 1/2
  s.soundOn = true; // not 0 => soundOn is true
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(1.0, 2.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch to B //////////

  s.audioSwitch = State::B;
  tester.device().setBool("/custom_properties/prop_audio_switch", s.audioSwitch);

  tester.nextFrame();

  ////////// Checks - srcB should now be the output //////////

  // led lights switched
  s.audioLEDA = false;
  s.audioLEDB = true;

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - turn on xfade //////////

  s.xFade = true;
  tester.device().setBool("/custom_properties/prop_xfade_switch", s.xFade);

  tester.nextFrame();

  ////////// Checks - same output (xfade only applies on switching) //////////

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch to A //////////

  s.audioSwitch = State::A;
  tester.device().setBool("/custom_properties/prop_audio_switch", s.audioSwitch);

  tester.nextFrame();

  ////////// Checks - srcA should now be the output but it should cross fade //////////

  // led lights switched
  s.audioLEDA = true;
  s.audioLEDB = false;

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, xfade(MockAudioDevice::buffer(3.0, 4.0), MockAudioDevice::buffer(1.0, 2.0)));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - disable xfade and wire cv (set to 0 = B) //////////
  // From documentation: Any positive (or 0) CV value will trigger the B input and any
  // negative CV value will trigger the A input.

  s.xFade = false;
  tester.device().setBool("/custom_properties/prop_xfade_switch", s.xFade);
  tester.wire(srcCV, "cvInAudio");
  srcCV->fValue = 0;

  tester.nextFrame();

  ////////// Checks - srcB should now be the output //////////

  // led lights switched
  s.audioLEDA = false;
  s.audioLEDB = true;
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch to B //////////

  s.audioSwitch = State::B;
  tester.device().setBool("/custom_properties/prop_audio_switch", s.audioSwitch);

  tester.nextFrame();

  ////////// Checks - srcB remains the output //////////

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch to A //////////

  s.audioSwitch = State::A;
  tester.device().setBool("/custom_properties/prop_audio_switch", s.audioSwitch);

  tester.nextFrame();

  ////////// Checks - srcB remains the output (CV override) //////////

  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch CV to negative (A) //////////

  srcCV->fValue = -1.0;

  tester.nextFrame();

  ////////// Checks - srcA is now the output //////////

  // led lights switched
  s.audioLEDA = true;
  s.audioLEDB = false;
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(1.0, 2.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we switch CV to positive (B) //////////

  srcCV->fValue = 1.0;

  tester.nextFrame();

  ////////// Checks - srcB is now the output //////////

  // led lights switched
  s.audioLEDA = false;
  s.audioLEDB = true;
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(3.0, 4.0));

  /*******************************////////////*******************************/
  /*                             < NEXT FRAME >                             */
  /*******************************////////////*******************************/
  ////////// Action - we disconnect CV //////////

  tester.unwire(srcCV);

  tester.nextFrame();

  ////////// Checks - srcA is now the output //////////

  // led lights switched
  s.audioLEDA = true;
  s.audioLEDB = false;
  ASSERT_EQ(s, State::from(tester));
  ASSERT_EQ(dst->fBuffer, MockAudioDevice::buffer(1.0, 2.0));


}
Top
User avatar
pongasoft
RE Developer
Posts: 478
Joined: 21 Apr 2016
Location: Las Vegas
Contact:
Contact pongasoft

21 Dec 2021

As an update on the project I have been adding a lot of things to the framework and I am extremely excited with the direction it is going. Support for patches, samples (loading and saving), fully load info.lua / motherboard_def.lua /realtime_controller.lua

Here is a very concrete example. I am working on a new RE (see Beta forum) and somebody reported a popping issue and provided a song to reproduce. I translated this into a unit test

Code: Select all

TEST(Device, PoppingBug)
{
  auto c = DeviceConfig<Device>::fromJBoxExport(RE_CMAKE_PROJECT_DIR)
    .patch_file("/Public/Bass Chug.repatch", "/tmp/Bass Chug.repatch");
    
  auto tester = InstrumentTester<Device>(c, 48000); // create the rack + device

  tester.wireMainOut("OutLeft", "OutRight"); // wire audio out

  tester.nextBatch(); // first batch (initialize the device)

  tester.device().loadPatch("/Public/Bass Chug.repatch"); // load the patch (saved from the Reason song)

  tester.nextBatch(); // apply patch

  // add midi notes
  tester.getSequencerTrack()
    .addNote(Midi::C(2),       sequencer::Time(1,1,1,0),   sequencer::Duration(0,0,0,90))
    .addNote(Midi::D_sharp(2), sequencer::Time(1,1,1,90),  sequencer::Duration(0,0,0,90))
    .addNote(Midi::G(2),       sequencer::Time(1,1,1,180), sequencer::Duration(0,0,0,90));

  // put breakpoint here
  auto sample = tester.play(sequencer::Duration(1)); // "play" the device for 1 bar

  tester.saveSample(sample, ConfigFile{"/tmp/PoppingBug.wav"}); // save the output to a file
Because the framework supports patches I could simply save the patch in Reason and load it in the test, making sure I was reusing all the properties with the exact same values without having to figure out what they were exactly. I then added 3 notes on the sequencer track (same position/duration as the Reason song). And then "play" for 1 bar (which will automatically trigger all the notes). I can then save the result and open it in Audacity to clearly see that there is pop (pretty obvious visually).

Once confirmed that the bug is reproducible in this very small unit test, since this runs straight in the IDE (CLion in my case) it is trivial to put a breakpoint and step through the code.

I should be able to release version 1.0 of the framework in the near future...
Top
Post Reply

Return to “RE Development and SDK Discussion”

  • Information

  • Who is online

    Users browsing this forum: No registered users and 2 guests