Advanced MIDI Thread - (DataBridge Release)

This forum is for discussing Propellerhead's music software. Questions, answers, ideas, and opinions... all apply.
User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 06 Aug 2019

EDIT:

For those unfamiliar, the DataBridge project is a control surface design that allows for direct device-to-device auto-modulation and distributed mass control of the rack using minimal hardware.

It was designed to add functionality to Reason that didn't exist at the time, as well as make creative expression easier to accomplish with less deliberation of manually programmed song structure.

Among the most advanced concepts discovered in the scope of this project were the non-linear smoothing of automation values and the interpolated recording technique known as audio warping / vinyl.

***

I gathered my materials and put them up on GitHub in the event that people want to explore the project.

I am continuing to flesh out the instructions in the background where and when I can. If there's interest, I can go into more depth on how it's used and other specifics.

GitHub
Last edited by Raveshaper on 14 Aug 2019, edited 2 times in total.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
JiggeryPokery
Posts: 1072
Joined: 15 Jan 2015

Post 06 Aug 2019

Raveshaper wrote:
06 Aug 2019
GitHub repo is up

I gathered my materials and put them up on GitHub in the event that people want to explore the project.

I am continuing to flesh out the instructions in the background where and when I can. If there's interest, I can go into more depth on how it's used and other specifics.

Here there be dragons
For the benefit of everyone who hasn't seen a thread or threads that may or may not be long buried, it might be useful for the thread title and the OP to actually state what you're talking about ;)

For example: What repo? What materials? What project? What interest in how something is used where, generally!?

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 06 Aug 2019

The project is back in active development, starting with debugging of the interpolated recording method.

For some reason there is a latching issue when editing or manipulating tempo speeds and I think the main cause for this is having more than one source of truth for how that implementation was achieved. I'll have to go through it again and see if I can pin down what the cause is.

Another thing I had not added by the time I got R9.5 is any support for the newer devices, like Players. I'll revisit them and build support.

I have a lot of irons in the fire at the moment so I'm doing a bit of juggling of projects. But it's been nagging at me that this was never properly finished or released. I welcome your questions, if there are any.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

CR68
Posts: 75
Joined: 17 Jan 2015

Post 06 Aug 2019

Thanks for your work. Now a stupid question: how can i Use this?

Thanks in advance for your help.

Cheers

User avatar
Boombastix
Posts: 904
Joined: 18 May 2018
Location: Bay Area, CA

Post 06 Aug 2019

Since I have Maschine Studio - should I be interested?
10% off at Waves with link: https://www.waves.com/r/6gh2b0
Disclaimer - I get 10% as well.

User avatar
Jagwah
Posts: 1560
Joined: 16 Jan 2015

Post 06 Aug 2019

Hey dude I remember you sharing an earlier version of this and while I appreciate your effort I think you need to look at it more from the perspective of a user who has no idea what you are presenting them with. I don't know what you mean when you say "device-to-device auto-modulation." I hope you aren't creating the terminology because that would be even more difficult for the end user. Instructions are definitely necessary, and you are going to need to use layman's terms.

Sterioevo
Posts: 97
Joined: 02 Apr 2015

Post 07 Aug 2019

If you follow the link provided you will find the information copied below (with formatting).

Very interesting idea btw
_______________

Data Bridge

Overview

Data Bridge is an advanced MIDI control surface designed to provide a scalable, modular network of responsive feedback connections between rack devices in Propellerhead Reason 7+.

User feedback was made hardware agnostic wherever possible, allowing for both legacy and modern hardware to be used interchangeably.

Why Was This Made?

The complexity of this design represents the best effort to facilitate solutions to the preference of creative spontaneity and immediacy of use that was at the forefront of discussion within the customer base of Reason 7 upon its release.

The specific areas that were addressed were:

Cumbersome interaction with MIDI hardware
Lack of support for live performance
Inflexible automation
Typical use of control surfaces in Reason involve binding a surface to a given device for the purpose of providing specific control.

Data Bridge is so-named because it allows the state of a given device's control data (knob values, etc.) to be broadcast to additional devices.

Applying MIDI data as a distributed network allows for unprecedented expressive control, as well as organically random auto-modulation.

Installation

Attach a physical MIDI 5-pin In and Out port to your computer.
Physically loop these ports with a MIDI cable (Out to In).
If Reason is running, exit the program.
Download this package locally.
Move the DataBridge folder in Codecs/Lua Codecs to the local Codecs/Lua Codecs folder used by your install of Reason.
Move the DataBridge folder in Maps to the local Maps folder used by your install of Reason.
For Help with 3 and 4, Please See: https://www.propellerheads.com/blog/control-remote
Start Reason.
Navigate to Preferences > Control Surfaces > Add and confirm that DataBridge appears as a manufacturer.
If this step fails, confirm folders were copied to the correct locations and restart Reason.
Exit Preferences.
Open the Local Workspace song project from the Templates and Patches folder in this package.
Expand the contents of the UI Device Combinator to reveal the External MIDI Instruments (EMI) within the patch.
Adjust the MIDI Port of all EMI units to reflect the physically looped port you attached in step 1.
NOTE: On Mac, select the port that does not contain the word "virtual".
Save the song project where you would like, then close the file and open a new blank project.
Navigate to Preferences > General and set the template song project you saved in step 13 as your default template for new projects.
Basic Usage

Upon successful installation, simply load a stock device and lock its dedicated control surface to it from the DataBridge list of surfaces.

NOTE: If the lock option isn't available, you will need to select Disable Master Keyboard first.
Once this initial surface is locked, you can bind its outputs to any other rack unit by selecting them from the list when assigning Remote Overrides.

You can scale this process to create control chains of arbitrary length according to your computer's specs.

IMPORTANT:

In order for data to propagate from one device to another, each device that is to send data out to a destination must have its own dedicated control surface locked to it.
A device receiving data from another device does not require a control surface by default.
Once all Remote Overrides have been mapped, you must prime the system by disconnecting your looped MIDI port, then plugging it back in.

Now simply activate the Clock On/Off button on the Combinator and adjust the control(s) whose outputs you mapped to a destination device.

You should see both controls moving even though you have only selected one.

NOTE: You can access items that cannot be controlled by external hardware as sources for outgoing data from a device, such as VU Meters, LEDs, and others.

Sterioevo
Posts: 97
Joined: 02 Apr 2015

Post 07 Aug 2019

Raveshaper wrote:
06 Aug 2019
EDIT:

For those unfamiliar, the DataBridge project is a control surface design that allows for direct device-to-device auto-modulation and distributed mass control of the rack using minimal hardware.

It was designed to add functionality to Reason that didn't exist at the time, as well as make creative expression easier to accomplish with less deliberation of manually programmed song structure.

Among the most advanced concepts discovered in the scope of this project were the non-linear smoothing of automation values and the interpolated recording technique known as audio warping / vinyl.

***

I gathered my materials and put them up on GitHub in the event that people want to explore the project.

I am continuing to flesh out the instructions in the background where and when I can. If there's interest, I can go into more depth on how it's used and other specifics.

Here there be dragons
Having spent a little time looking at the files and work involved all I can say is well done and thank you, this is a very generous offering. There is a lot to digest and learn from.

I noticed that you haven't mapped "select next(previous) patch". Is it possible to map these parameters that can then be chained to other devices? This would allow a function similar to program changes across devices, currently not possible in reason.

In an attempt to simplify what this device does here is my take.... This is a system of virtual remote mapping that allows you to "hijack" the remote message going out (remote is 2 way communication) and connect this to another parameter on another device via remote override. For example, when you have your controller keyboard connected to thor (setup with 2 way communication) and you have a visual display, this will be updated if you use the mouse to adjust the Filter Frequency - it is this midi information feedback provided by the remote communication that can then be connected to other parameters via remote overrides. Typically you do not have access this information in this way.

I am yet to give it a proper workout in practice, I am interested to see how it performs.

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

Sterioevo wrote:This is a system of virtual remote mapping that allows you to "hijack" the remote message going out (remote is 2 way communication) and connect this to another parameter on another device via remote override.
Better Explanation

That is basically what's happening, but slightly different than the way it's stated above. I'll do my best to explain it as best I can:

This is a system that copies the values of a device's inputs (controls), then sends those values to other device inputs in the form of mappable Remote Overrides.

In short, this system allows the user to chain together the front panels of devices so the rack is manipulated on a mass scale.

Complexity

What makes this design so complex is that the number of outputs per front panel control is customizable and granular down to each stock device type. This means Subtractors can have a specific number of these outputs compared to, say, a Scream4.

To make matters even more complex, each of the output Remote Override mappings can be individually edited to have unique behavior; such as a minimum and maximum value, unipolar or bipolar response, rotation polarity (reverse), variable speed sensitivity to endless encoders, and interpolation between values using a custom wavetable.

It's not just that you can make chains, but that each point along the chain will behave differently than the others (if you so choose).

Patches?

To answer the question about the patch loading options, sadly this isn't possible because they take TextInput. This is a data type that cannot be sent over the Remote I/O protocol and is actually sandboxed to just within Reason itself.

As small consolation, you can take advantage of an imitation bash terminal that uses 63 characters. This is accessed by entering commands in the device name tag of each instrument or effect that is locked to one of the surfaces in the system. Its primary purpose is to provide powerful editing tools to make it easier for the user to set things up exactly as they want them to be.

The terminal and the ability to bind key commands to puppet the UI Device combinator are basically the bleeding edge of user experience in this thing. Well, that and one last thing.

VSTs

You can use this to provide a sort of modulation routing to VSTs similar to how Massive does it.

***

I will do my best to get images and better docs made of how to get all this working in practice. I hope it proves useful to some insane patchers and modheads out there.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

CR68 wrote:
06 Aug 2019
Thanks for your work. Now a stupid question: how can i Use this?
Thanks for the thanks and no question is stupid.

I will be updating the docs between now and Monday, 12 August.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

Boombastix wrote:
06 Aug 2019
Since I have Maschine Studio - should I be interested?
I personally use a Maschine Mk2 so this is definitely something you might want to try out.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

Disclaimer Time

I need to mention these before we get too deep into this.

1. Undo becomes broken when using this system

Reason sees all of the adjustments being made as user events, so Undo is essentially worthless when it's running.

2. Reason will not save your custom parameters

Sadly you will need to actually note down specific configs in order to recreate them if you close the project before finishing it.

You can develop strategies to address these, but there is very much an element of improvisation and bravery involved.

That said, it's very much worth it.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Jagwah
Posts: 1560
Joined: 16 Jan 2015

Post 08 Aug 2019

I never really understood it but it seems you are well on your way. If it can help us with live sessions that will be fantastic.

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

Jagwah wrote:
08 Aug 2019
I never really understood it but it seems you are well on your way. If it can help us with live sessions that will be fantastic.
Song Position and Loop left/right can be manipulated with this, plus there's support for 2 deck setups and scenes/pages for controllerists.

I tried to think of everything.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

botnotbot
Posts: 257
Joined: 26 Oct 2017

Post 08 Aug 2019

This is a pretty brilliant hack Raveshaper. It's going to take me some time to wrap my head around this, so I'm putting it off for a better time.

That said, I'd like to make a suggestion: maybe some example setups using the free Hamu plugins? I was barely around before he passed away but I get the sense from his devices that he would be thrilled by this project if he were still with us.

My reasoning is that a lot of his work supplies plenty of knobs and particularly ModPanel is a very easy conceptual leap to mapping onto controllers. I think he has some momentary switches that would work for mapping pads in some of the other devices.

Anyway, just a thought. Thanks for this epic contribution to the world of remote mapping/controlling Reason!

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 08 Aug 2019

botnotbot wrote:
08 Aug 2019
Maybe some example setups using the free Hamu plugins? I was barely around before he passed away but I get the sense from his devices that he would be thrilled by this project if he were still with us.

My reasoning is that a lot of his work supplies plenty of knobs and particularly ModPanel is a very easy conceptual leap to mapping onto controllers. I think he has some momentary switches that would work for mapping pads in some of the other devices.
I had no idea about Hamu.

I can certainly try to provide support for the Hamu extensions. It depends on how much the devices support Remote. If they're fully unlockable, it will be somewhat of a time consuming process. I can make setups and demos based on what I have so far until then.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

botnotbot
Posts: 257
Joined: 26 Oct 2017

Post 08 Aug 2019

Raveshaper wrote:
08 Aug 2019
botnotbot wrote:
08 Aug 2019
Maybe some example setups using the free Hamu plugins? I was barely around before he passed away but I get the sense from his devices that he would be thrilled by this project if he were still with us.

My reasoning is that a lot of his work supplies plenty of knobs and particularly ModPanel is a very easy conceptual leap to mapping onto controllers. I think he has some momentary switches that would work for mapping pads in some of the other devices.
I had no idea about Hamu.

I can certainly try to provide support for the Hamu extensions. It depends on how much the devices support Remote. If they're fully unlockable, it will be somewhat of a time consuming process. I can make setups and demos based on what I have so far until then.
:thumbs_up: :thumbs_up: :thumbs_up:

Steady on, sounds like a good plan. When I finally get a chance to figure it out, I can look at lending a hand.

User avatar
boingy
Posts: 791
Joined: 01 Feb 2019

Post 08 Aug 2019

Hooray! More complexity! Woo hoo! etc.

/sarcasm

User avatar
mjxl
Posts: 360
Joined: 23 Nov 2018

Post 08 Aug 2019

Raveshaper wrote:
08 Aug 2019
I had no idea about Hamu.

I can certainly try to provide support for the Hamu extensions. It depends on how much the devices support Remote. If they're fully unlockable, it will be somewhat of a time consuming process. I can make setups and demos based on what I have so far until then.
Yep HaMu devices are fully remotable as far as i could tell.

Also super interesting project, there's a lot to digest. Time to put my learning cap on..

I'm interested in the ability to split remotemaps per device, like you said (in the request topic) it can be quite cumbersome to load remotemaps with an insane amount of lines and then figure out things.
:reason: 10+ :re:

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 09 Aug 2019

Just a tiny update.

I finally got the audio warp engine debugged (tentatively) so I will be working on a demo of that next before fleshing out the docs. I've been wanting to use it ever since I broke it.

10 octaves of signal mangling pitch warp incoming soon.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 14 Aug 2019

I have a lot more of the setup documentation written for the system, complete with images to help reduce the confusion level. It's taking a lot more time to write because there's so much ground to cover once I walk through it from a novice perspective. I have the unusual luxury (or curse) of having built this thing, but now I have to explain it. Not as easy as I had thought it would be.

Because of this, no demos of sounds yet. If I make the demos, people might want to use it, but that doesn't do any good if there isn't a way of understanding how to use it. At least that's the logic behind that decision. Feel free to tell me if my thinking is flawed on that.

There are six sections left to write for the documentation, each with their own levels of complexity. I don't think many will use the most advanced features so I'll save those for last. To be frank, every aspect of this is "advanced" as it is.

Here's the most informative image about how the system works, so far.
Image
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
demt
Posts: 875
Joined: 16 Sep 2016

Post 18 Aug 2019

video of it working?
neutron; tb3; triggerfingerpro;3korg volca keys fm drums; kaos pad 3;zaquencer; akai mpk mini;2 zmx mixers; midiplus interface;16 channel behringer umc 1820 with ada8200 soundcard, arturia drumbrute.nektar p1,
hear scince reason 2.5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 18 Aug 2019

This project is only one of several I'm working on right now so this may take some time.

Since it's already completed and only needs documentation it isn't top priority. To be perfectly honest, I am a bit eager to close the book on this and conclude what has been a bitter sweet journey. An appropriate way to do that would be a video and audio demonstration of just what the hell this thing is all about.

I'll try to set aside time for recording a video or two.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 21 Aug 2019

Just updated the documentation to include the command terminal functionality.

You can use that feature set to create keyboard splits and page/scene bindings for instruments, as well as enable or disable master keyboard input as desired. Endless rotary encoder support for coarse/fine adjustments based on rotation speed is also enabled using the terminal.

At this point, the entire documentation is pretty much there.
Videos on how to use it are still TBA.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

User avatar
Raveshaper
Posts: 1079
Joined: 16 Jan 2015

Post 30 Aug 2019

Docs still need some love, but that will take a bit of time for me to get to.

Right now I think that videos will speak louder than text.

Some small good news: IAC Driver can be used as a super fast "in the box" loopback port. Meaning you don't have to go through all the big long "plug in a physical port" stuff with a cable and a controller, etc. It's just easier to jumpstart things by unplugging and reconnecting your USB midi gear if there's a problem.

What I really want to know at this point is:

What kind of video demos do you want?
  • See and hear it in action (prepared project with system running)
  • Watch the workflow (process of building sounds)
  • Narrated walkthrough (me discussing things as I go)
I need to figure out how to get setup for screen capture and production quality will probably be pretty low given my current setup, but I need to get demos of this out there before the sun goes nova.
:reason: :ignition: :re: :refillpacker: Enhanced by DataBridge v5

  • Information
  • Who is online

    Users browsing this forum: CommonCrawl [Bot], dhruan, mjxl, Trendiction [Bot] and 4 guests