Python to .repatch (Mutator)

This forum is for developers of Rack Extensions to discuss the RE SDK, share code, and offer tips to other developers.
User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 29 Sep 2023

That's good, as it's driving me crazy at the moment lol

I've updated the script to read PM string and it seems to work ok :-

Code: Select all

def mutator_schema(hex_string):

    version = int(hex_string[:2], 16)

    msb = hex_string[2:4]
    lsb = hex_string[4:6]
    msb_decimal = int(msb, 16)
    lsb_decimal = int(lsb, 16)
    total_notes = (msb_decimal - 1) * 256 + lsb_decimal - 1

    note_data = []

    for i in range(len(hex_string) // 12):
        step = int(hex_string[6 + i * 12:8 + i * 12], 16)
        pitch = int(hex_string[8 + i * 12:10 + i * 12], 16) - 1
        velocity = int(hex_string[10 + i * 12:12 + i * 12], 16) - 1
        lengthinsteps = int(hex_string[12 + i * 12:14 + i * 12], 16) - 1
        flags = int(hex_string[14 + i * 12:16 + i * 12], 16)
        mute = bool(flags & 32)
        tied = bool(flags & 16)
        octave_offset = flags & 7 - 3

        note_data.append((step, pitch, velocity, lengthinsteps, mute, tied, octave_offset))

    return version, total_notes, note_data

# sample data
#pattern_data_1 = "020110013D67028380023D5E028380033D63028380043D35028380053D6A028380063D54028380073D36028380083D69028380093D300283800A3D2E0283800B3D610283800C3D320283800D3D670283800E3D340283800F3D43028380"
pattern_data_1 = "020108013D65118380113D65098380193D650583801D3D650383801F3D65028380203D65028380213D65028380"

version, total_notes, note_data = mutator_schema(pattern_data_1)

print(f'{version} {total_notes}')
for note in note_data:
    step, pitch, velocity, lengthinsteps, mute, tied, octave_offset = note   
    print(f"Note: pos={step:02}, pitch={pitch}, velocity={velocity:03}, length={lengthinsteps}, mute={mute}, tied={tied}, octave_offset={octave_offset}")


but I've got a bug ( probably more than one ) in my script to build from midi..

Code: Select all

import sys
from mido import MidiFile

def warningnotice():
    try:
        print("This script is almost certainty broken and should only be used if you accept responsibility")
        
        agree = input("Do you argee (y/n): ")
        if agree.lower() == "n":
            print("Exiting the program...")
            sys.exit(0)
        
    except Exception as e:
        print(f"An error occurred: {e}")
        sys.exit(1)


ppqn = 0
whole_note_duration = 0
notescalefactor = 254
version = '02' 
note_count = 0
max_notes = 340
flags = '8380'  # dont think i need to worry about this, so long as its added.
offset = 1
pattern_data = []
pattern_data_1 = ''


def calculate_standard_notation(note_duration):
    # Calculate the length of the note in standard notation
    standard_notation = ""
    if note_duration == whole_note_duration:
        standard_notation = "Whole Note"
    elif note_duration == whole_note_duration / 2:
        standard_notation = "1/2 Note"
    elif note_duration == whole_note_duration / 4:
        standard_notation = "1/4 Note"
    elif note_duration == whole_note_duration / 8:
        standard_notation = "1/8 Note"
    elif note_duration == whole_note_duration / 16:
        standard_notation = "1/16 Note"
    elif note_duration == whole_note_duration / 32:
        standard_notation = "1/32 Note"
    elif note_duration == whole_note_duration / 64:
        standard_notation = "1/64 Note"
    elif note_duration == whole_note_duration / 128:
        standard_notation = "1/128 Note"
    
    return standard_notation


def calculate_totalnotes_value(integer):
    msb = (integer >> 8) & 0xFF
    lsb = integer & 0xFF
    msb += offset
    lsb += offset
    total_notes_hex = f"{msb:02X}{lsb:02X}"

    return total_notes_hex


def read_midi_file(file_path):
    mid = MidiFile(file_path)
    ppqn = mid.ticks_per_beat
    global whole_note_duration
    whole_note_duration = ppqn * 4
    global note_count
    pos = 1
    note_length = 1
    for i, track in enumerate(mid.tracks):
        if track.name > '':
            print(f'Track [{i:03}] Name: {track.name}')
        for j, msg in enumerate(track):
            if note_count == max_notes:
                        print('max note count reached')
                        break
            if msg.type == "note_on":
                note_on_time = msg.time
                note_off_time = track[j + 1].time
                note_duration = note_off_time - note_on_time
                standard_notation = calculate_standard_notation(note_duration)
                if standard_notation >"":
                    note_count += 1
                    
                    if standard_notation == "Whole Note":
                        note_length = 16
                    elif standard_notation == "1/2 Note":
                        note_length = 8
                    elif standard_notation == "1/4 Note":
                        note_length = 4
                    elif standard_notation == "1/8 Note":
                        note_length = 2
                    elif standard_notation == "1/16 Note":
                        note_length = 1
                    elif standard_notation == "1/32 Note":
                        note_length = 1
                    elif standard_notation == "1/64 Note":
                        note_length = 1
                    elif standard_notation == "1/128 Note":
                        note_length = 1
                    
                    # still note sure about scale factor and what value it is used for?
                    print(f'Note [{note_count:03}] ON  Pos [{pos:03}] Length [{note_length:02}] Factor {int(note_duration / notescalefactor):02X}')

                    pattern_data.append((pos, msg.note, msg.velocity, note_length, flags))

                    pos += note_length

                    # this is now adding an extra patterdata note ? i think i broke the logic......
            if msg.type == 'note_off':
                print(f'Note [{note_count:03}] OFF Pos [{pos:03}] ')

    global pattern_data_1
    
    steps = calculate_totalnotes_value(note_count)
    pattern_data_1 = f'{version}{steps}'
    print(f'PM Version {version} Note Count {steps}')
    for k, data in enumerate(pattern_data):
        pos, note, velocity, length, flag = data
        pattern_data_1 += f'{pos:02X}{note+offset:02X}{velocity+offset:02X}{length+offset:02X}{flag}'
        print(f'[{k+1:02}] {pos:02X} {note+offset:02X} {velocity+offset:02X} {length+offset:02X} {flag}')

    print(f'\n{pattern_data_1}')


if len(sys.argv) > 1:
    # i'm still working on this, so there is defo bugs....
    warningnotice()
    read_midi_file(sys.argv[1])
else:
    print("Please provide a midi filename.")


User avatar
jam-s
Posts: 3128
Joined: 17 Apr 2015
Location: Aachen, Germany

Post 29 Sep 2023

To get your script to the next level I think you should take a look into data classes. Those would be the most natural way to represent individual notes and also collections of those (aka patterns). Going to a more object oriented code style might also help in finding bugs and writing unittests for your code.

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 29 Sep 2023

jam-s wrote:
29 Sep 2023
To get your script to the next level I think you should take a look into data classes. Those would be the most natural way to represent individual notes and also collections of those (aka patterns). Going to a more object oriented code style might also help in finding bugs and writing unittests for your code.
ok well I've posted my current testing script, do you think you could advance it in anyway as currently my few weeks programming in python has left me stumped to the point i'm currently rewriting from the ground up......

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 30 Sep 2023

I can't get my head around midi files that don't include note_off messages but instead have note_on with velocity=00.

if I stick with my 16 step H M S R I'm sorted but dealing with someone else's midi file causes my head to hurt :lol:

User avatar
buddard
RE Developer
Posts: 1263
Joined: 17 Jan 2015
Location: Stockholm

Post 01 Oct 2023

Billy+ wrote:
29 Sep 2023
I've updated the script to read PM string and it seems to work ok
Just a minor detail: You have to multiply the msb with 255. This is because we can’t have any 00 bytes in the string, so there are only 255 possible values for each byte.

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 01 Oct 2023

Progress report:

Got most of my basic functions worked out this afternoon after spending most of my free time this weekend converting my wife’s rarely used laptop into a mobile dev station! 😊😊😊

Still need to build the script itself but from here on in it’s child’s play, and my functions will be reusable for other such scripts.

FWIW, I’m just doing a simple version that will spit out a thousand random PM patches in a directory, where each Pattern 1 is data filled and the rest empty so that they are actually of use within the realm of PM - having all of them filled would be a bit counter productive IMHO.

I also need to state that I see little advantage to generating a random MIDI file and then converting that to a PM format. However, if you are doing this from a rhythmic perspective then I could add an optional args set that allows you to predefine the note range. Would this kill both birds with one stone?

Cheers,
Brian

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 01 Oct 2023

buddard wrote:
01 Oct 2023
Just a minor detail: You have to multiply the msb with 255. This is because we can’t have any 00 bytes in the string, so there are only 255 possible values for each byte.
Hi Buddard,

Thanks for all your help with this! 😊

For clarity, I read the above to say that a string with 255 events would have a flag of “0201”, is that correct?

Cheers,
Brian

User avatar
buddard
RE Developer
Posts: 1263
Joined: 17 Jan 2015
Location: Stockholm

Post 01 Oct 2023

Enlightenspeed wrote:
01 Oct 2023
buddard wrote:
01 Oct 2023
Just a minor detail: You have to multiply the msb with 255. This is because we can’t have any 00 bytes in the string, so there are only 255 possible values for each byte.
Hi Buddard,

Thanks for all your help with this! 😊

For clarity, I read the above to say that a string with 255 events would have a flag of “0201”, is that correct?

Cheers,
Brian
Yes, that is correct!

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 01 Oct 2023

buddard wrote:
01 Oct 2023
Enlightenspeed wrote:
01 Oct 2023


Hi Buddard,

Thanks for all your help with this! 😊

For clarity, I read the above to say that a string with 255 events would have a flag of “0201”, is that correct?

Cheers,
Brian
Yes, that is correct!
Great stuff,
Thanks again,
Brian

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 05 Oct 2023

Enlightenspeed wrote:
01 Oct 2023
Progress report:

Got most of my basic functions worked out this afternoon after spending most of my free time this weekend converting my wife’s rarely used laptop into a mobile dev station! 😊😊😊

Still need to build the script itself but from here on in it’s child’s play, and my functions will be reusable for other such scripts.

FWIW, I’m just doing a simple version that will spit out a thousand random PM patches in a directory, where each Pattern 1 is data filled and the rest empty so that they are actually of use within the realm of PM - having all of them filled would be a bit counter productive IMHO.

I also need to state that I see little advantage to generating a random MIDI file and then converting that to a PM format. However, if you are doing this from a rhythmic perspective then I could add an optional args set that allows you to predefine the note range. Would this kill both birds with one stone?

Cheers,
Brian
that all sounds good to me, although I would maybe not spit out 1000 at a time, but that's for you I guess as it sound like you've made way more progress than me.

can wait to get my eyes on the code ;)

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 05 Oct 2023

buddard wrote:
01 Oct 2023
Billy+ wrote:
29 Sep 2023
I've updated the script to read PM string and it seems to work ok
Just a minor detail: You have to multiply the msb with 255. This is because we can’t have any 00 bytes in the string, so there are only 255 possible values for each byte.
was this in ref to the 1st post or 2nd post of the reader?

User avatar
buddard
RE Developer
Posts: 1263
Joined: 17 Jan 2015
Location: Stockholm

Post 05 Oct 2023

Billy+ wrote:
05 Oct 2023
buddard wrote:
01 Oct 2023


Just a minor detail: You have to multiply the msb with 255. This is because we can’t have any 00 bytes in the string, so there are only 255 possible values for each byte.
was this in ref to the 1st post or 2nd post of the reader?
I'm sorry, but I don't understand your question?

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 10 Oct 2023

ok so I've still got stuff going on that is leaving me with not a lot of time at the keyboard, However I've made progress with processing ANY midi files and have managed to generate acceptable output for all my files that previously caused errors. long story short I needed to use this :- https://github.com/music-being/miditoolkit as the pip install miditoolkit is broken.

Code: Select all

import sys
from miditoolkit.midi import parser as mid


def is_midi_file(filename):
    try:
        midi_file = mid.MidiFile(filename)
        return True
    except:
        return False


def get_note_name(note_number):
    note_names = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
    return note_names[note_number % 12]


def get_octave_number(note_number):
    return (note_number // 12) - 1


def parse_midifile(filename):
    mididata = mid.MidiFile(filename)
    notecount = 1
    for Note in mididata.instruments[0].notes:
        note_length = Note.end - Note.start
        note_name = get_note_name(Note.pitch)
        octave_number = get_octave_number(Note.pitch)

        print(f'[{notecount:03}] [{note_name}{octave_number} | Pitch {Note.pitch}] Velocity [{Note.velocity}] Start [{Note.start}] End [{Note.end}] Duration [{note_length}]')
        notecount += 1


if __name__ == '__main__':
    filename = sys.argv[1]
    max_notes = 340
    
    if is_midi_file(filename):
        parse_midifile(filename)
       
    else:
        print(f"{filename} is not a valid MIDI file.")
        
was kind of hoping Enlightenspeed had posted a better method to produce pattern_data_1 but I'm sure he's got less time than me... I'm still hoping to move forward with this not so little project so any input would be great, as I'm sure there's way more to it than I realize :oops:

User avatar
buddard
RE Developer
Posts: 1263
Joined: 17 Jan 2015
Location: Stockholm

Post 11 Oct 2023

A key part of the MIDI to pattern conversion is of course quantization, since PM is a step based sequencer.

The simplest approach is to create the pattern at a fixed resolution (either hardcoded in the converter or specified by the user via command line switch) and quantize the MIDI to that. Another approach is to analyze the MIDI to determine the best resolution within the limits of Mutator (i e what is the lowest resolution needed to capture the MIDI data, and does it fit within the 128 step limit).

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 14 Oct 2023

Billy+ wrote:
10 Oct 2023
was kind of hoping Enlightenspeed had posted a better method to produce pattern_data_1 but I'm sure he's got less time than me...
I do indeed have very little time right now, but I'm hoping to get at least a first draft up before the end of this weekend. :) This will not be the prettiest code you'll ever see as it is fully procedural and could use some refactoring just to make it look less bulky. But it's almost done.

Back soon,
Brian

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 15 Oct 2023

well if you've looked at any of mine it's not exactly golden ;) and i'm still very much having issues with 3rd party midi files.

here is a few examples with my current testing script

Code: Select all

PS C:\billy> python .\roundres.py '.\midifiles\Ain''t Nobody.mid' | more
PPQN 384 BPM 104.0005546696249 Track A.PIANO 2: extended note data:
{'pos': 1, 'pitch': 63, 'velocity': 113, 'start': 1920, 'end': 1958, 'duration': 38, 'timetonext': -38, 'resolution': (7130699410003285, 288230376151711744)}
{'pos': 2, 'pitch': 66, 'velocity': 120, 'start': 1920, 'end': 1958, 'duration': 38, 'timetonext': -38, 'resolution': (7130699410003285, 288230376151711744)}
{'pos': 3, 'pitch': 58, 'velocity': 113, 'start': 1920, 'end': 1964, 'duration': 44, 'timetonext': 148, 'resolution': (8256599316845909, 288230376151711744)}
{'pos': 4, 'pitch': 63, 'velocity': 120, 'start': 2112, 'end': 2146, 'duration': 34, 'timetonext': -34, 'resolution': (6380099472108203, 288230376151711744)}
{'pos': 5, 'pitch': 66, 'velocity': 120, 'start': 2112, 'end': 2154, 'duration': 42, 'timetonext': -42, 'resolution': (7, 256)}
{'pos': 6, 'pitch': 58, 'velocity': 120, 'start': 2112, 'end': 2160, 'duration': 48, 'timetonext': 144, 'resolution': (1, 32)}
{'pos': 7, 'pitch': 61, 'velocity': 120, 'start': 2304, 'end': 3080, 'duration': 776, 'timetonext': -776, 'resolution': (4550512123488939, 9007199254740992)}
{'pos': 8, 'pitch': 56, 'velocity': 120, 'start': 2304, 'end': 3092, 'duration': 788, 'timetonext': -788, 'resolution': (4620880867666603, 9007199254740992)}
{'pos': 9, 'pitch': 65, 'velocity': 118, 'start': 2304, 'end': 3096, 'duration': 792, 'timetonext': 360, 'resolution': (33, 64)}
{'pos': 10, 'pitch': 63, 'velocity': 120, 'start': 3456, 'end': 3500, 'duration': 44, 'timetonext': -44, 'resolution': (8256599316845909, 288230376151711744)}
{'pos': 11, 'pitch': 66, 'velocity': 120, 'start': 3456, 'end': 3500, 'duration': 44, 'timetonext': -44, 'resolution': (8256599316845909, 288230376151711744)}
{'pos': 12, 'pitch': 58, 'velocity': 113, 'start': 3456, 'end': 3500, 'duration': 44, 'timetonext': 148, 'resolution': (8256599316845909, 288230376151711744)}
{'pos': 13, 'pitch': 63, 'velocity': 120, 'start': 3648, 'end': 3682, 'duration': 34, 'timetonext': -34, 'resolution': (6380099472108203, 288230376151711744)}
{'pos': 14, 'pitch': 58, 'velocity': 100, 'start': 3648, 'end': 3688, 'duration': 40, 'timetonext': -40, 'resolution': (7505999378950827, 288230376151711744)}

PS C:\billy> python .\roundres.py '.\midifiles\A Whiter Shade Of Pale.mid' | more
PPQN 384 BPM 72.00002880001152 Track : extended note data:
{'pos': 1, 'pitch': 36, 'velocity': 117, 'start': 3072, 'end': 3552, 'duration': 480, 'timetonext': 96, 'resolution': (5, 16)}
{'pos': 2, 'pitch': 36, 'velocity': 117, 'start': 3648, 'end': 3840, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}
{'pos': 3, 'pitch': 35, 'velocity': 114, 'start': 3840, 'end': 4320, 'duration': 480, 'timetonext': 96, 'resolution': (5, 16)}
{'pos': 4, 'pitch': 35, 'velocity': 117, 'start': 4416, 'end': 4608, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}
{'pos': 5, 'pitch': 33, 'velocity': 109, 'start': 4608, 'end': 5184, 'duration': 576, 'timetonext': 0, 'resolution': (3, 8)}
{'pos': 6, 'pitch': 33, 'velocity': 105, 'start': 5184, 'end': 5376, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}
{'pos': 7, 'pitch': 31, 'velocity': 108, 'start': 5376, 'end': 5856, 'duration': 480, 'timetonext': 96, 'resolution': (5, 16)}
{'pos': 8, 'pitch': 31, 'velocity': 113, 'start': 5952, 'end': 6048, 'duration': 96, 'timetonext': 96, 'resolution': (1, 16)}
{'pos': 9, 'pitch': 41, 'velocity': 114, 'start': 6144, 'end': 6624, 'duration': 480, 'timetonext': 96, 'resolution': (5, 16)}
{'pos': 10, 'pitch': 41, 'velocity': 105, 'start': 6720, 'end': 6912, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}
{'pos': 11, 'pitch': 40, 'velocity': 108, 'start': 6912, 'end': 7488, 'duration': 576, 'timetonext': 0, 'resolution': (3, 8)}
{'pos': 12, 'pitch': 40, 'velocity': 101, 'start': 7488, 'end': 7680, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}
{'pos': 13, 'pitch': 38, 'velocity': 103, 'start': 7680, 'end': 8256, 'duration': 576, 'timetonext': 0, 'resolution': (3, 8)}
{'pos': 14, 'pitch': 38, 'velocity': 95, 'start': 8256, 'end': 8448, 'duration': 192, 'timetonext': 0, 'resolution': (1, 8)}

PS C:\billy> python .\roundres.py .\temp\PM-Templates\16th.mid | more
PPQN 15360 BPM 120.0 Track ID8 1: extended note data:
{'pos': 1, 'pitch': 60, 'velocity': 100, 'start': 0, 'end': 3840, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 2, 'pitch': 60, 'velocity': 100, 'start': 3840, 'end': 7680, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 3, 'pitch': 60, 'velocity': 100, 'start': 7680, 'end': 11520, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 4, 'pitch': 60, 'velocity': 100, 'start': 11520, 'end': 15360, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 5, 'pitch': 60, 'velocity': 100, 'start': 15360, 'end': 19200, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 6, 'pitch': 60, 'velocity': 100, 'start': 19200, 'end': 23040, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 7, 'pitch': 60, 'velocity': 100, 'start': 23040, 'end': 26880, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 8, 'pitch': 60, 'velocity': 100, 'start': 26880, 'end': 30720, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 9, 'pitch': 60, 'velocity': 100, 'start': 30720, 'end': 34560, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 10, 'pitch': 60, 'velocity': 100, 'start': 34560, 'end': 38400, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 11, 'pitch': 60, 'velocity': 100, 'start': 38400, 'end': 42240, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 12, 'pitch': 60, 'velocity': 100, 'start': 42240, 'end': 46080, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 13, 'pitch': 60, 'velocity': 100, 'start': 46080, 'end': 49920, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}
{'pos': 14, 'pitch': 60, 'velocity': 100, 'start': 49920, 'end': 53760, 'duration': 3840, 'timetonext': 0, 'resolution': (1, 16)}


which is still leaving me scratching my head a little, with not much time during the week and needed to get my head in the game after putting it down for days at a time it almost feels like i'm starting fresh eveytime i open the ide ..

my current thining is that i need to round a few of the durations and quant a few starts but i was never very good a math lol

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 16 Oct 2023

Hi Billy,

I've been a bit ill on Sunday, due to much beer and a bucket load of stupidly hot'n'spicy chicken on Saturday. Thankfully I'm kinda quiet in the "real work" office today, so I'm likely to get this finished on the skive by about 4 o'clock UK time.

I can't much help with what you're doing as I'm not building MIDI files, I'm just doing a generator for Pattern Mutator, but it'll be easy to edit the details for it. I don't really know what I'm looking at with your MIDI file stuff, I get the general concept, but it seems a bit counter intuitive to build MIDI files from a randomiser program and then port them into Reason when you can build the random files for Pattern Mutator and others quite easily, by doing it more directly.

This being the case, is there some usage that I'm missing for these MIDI files? You see MIDI files are a kind of ugly old beast, and modern DAWs support them only really as a legacy thing and have built frameworks for making better and more musical output, an example being the Players format in Reason/RRP. Basically, what is the motivation for doing things the way you are trying to do it?

Cheers,
Brian (another ugly old beast)

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 16 Oct 2023

Enlightenspeed wrote:
16 Oct 2023
Hi Billy,

I've been a bit ill on Sunday, due to much beer and a bucket load of stupidly hot'n'spicy chicken on Saturday. Thankfully I'm kinda quiet in the "real work" office today, so I'm likely to get this finished on the skive by about 4 o'clock UK time.

I can't much help with what you're doing as I'm not building MIDI files, I'm just doing a generator for Pattern Mutator, but it'll be easy to edit the details for it. I don't really know what I'm looking at with your MIDI file stuff, I get the general concept, but it seems a bit counter intuitive to build MIDI files from a randomiser program and then port them into Reason when you can build the random files for Pattern Mutator and others quite easily, by doing it more directly.

This being the case, is there some usage that I'm missing for these MIDI files? You see MIDI files are a kind of ugly old beast, and modern DAWs support them only really as a legacy thing and have built frameworks for making better and more musical output, an example being the Players format in Reason/RRP. Basically, what is the motivation for doing things the way you are trying to do it?

Cheers,
Brian (another ugly old beast)
In the grand scheme of things there's no reason to process midi this way and I managed to get my generated stuff to work mostly until I decided to play with 3rd party midi files, and your right there very ugly beasts. it makes far more sense to just load the midi file in reason and record the section into PM.

I'm just using it as an excuse to do more python than I need to, it's a long term method of going bald :lol:

look forward to checking out your code :thumbup: as for your hangover my only suggestion is to never be soba its the best solution I've found over the years and spicy chicken is almost defiantly the best mid drink snack ;)

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 16 Oct 2023

can anyone tell me if the following is correct ?

Code: Select all

def generate_resolutions():
    # generate resolution duration values.
    resolutions = {
    "1/1": ppqn*4, 
    "1/2": round((ppqn*4) /2),
    "1/4": round((ppqn*4) /4), 
    "1/8": round((ppqn*4) /8),
    "1/16": round((ppqn*4) /16),
    "1/16t": round(((ppqn*4) /16) * (2/3)), 
    "1/32": round((ppqn*4) /32),
    "1/32t": round(((ppqn*4) /32) * (2/3)),
    "1/64": round((ppqn*4) /64),
    "1/64t": round(((ppqn*4) /64) * (2/3)),
    "1/128": round((ppqn*4) /128)
    }

    return resolutions

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 16 Oct 2023

Righties,

consider this a first draft, as I had to go for a snooze earlier and I'm still not feeling great.
Pat_The_Mutt.zip


This gives you random patterns in the first pattern slot, with the rest empty.

At this point there is no accents or cv lanes, and the note start points (if using the key modes) are not in there yet, it's just the patterns themselves. I'll fix some of this tomorrow and re-upload, then a little more the next day etc.

You can see that a lot of refactoring is needed, but the truth is you can't do as much as would be ideal. I'm posting this up now for Billy's benefit and then it's easier for him and any others to watch what is being changed when I refactor it. For this reason I'm also not using Beautiful Soup, Lxml etc. I may demonstrate this kind of thing later. For now it's simple and strictly procedural. I'll also improve the docs later (it's "always later" for the docs with me :) ).

To use this, and any future versions you need to install Python 3, and then set the .py file type to be executed by Python 3, by right-clicking on the code file, selecting "Properties", and in the general menu click on "Opens with" and select Python. To produce a directory full of patches you can then just double-click on it, no CLI is needed - unless you like that sort of thing, of course. Open the file in your IDE of choice should you wish to examine the code.

I wouldn't have thought this code was too hard to follow, but if you struggle, let me know what with and I'll try and explain.

cheers,
Brian
You do not have the required permissions to view the files attached to this post.

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 16 Oct 2023

Billy+ wrote:
16 Oct 2023
can anyone tell me if the following is correct ?

Code: Select all

def generate_resolutions():
    # generate resolution duration values.
    resolutions = {
    "1/1": ppqn*4, 
    "1/2": round((ppqn*4) /2),
    "1/4": round((ppqn*4) /4), 
    "1/8": round((ppqn*4) /8),
    "1/16": round((ppqn*4) /16),
    "1/16t": round(((ppqn*4) /16) * (2/3)), 
    "1/32": round((ppqn*4) /32),
    "1/32t": round(((ppqn*4) /32) * (2/3)),
    "1/64": round((ppqn*4) /64),
    "1/64t": round(((ppqn*4) /64) * (2/3)),
    "1/128": round((ppqn*4) /128)
    }

    return resolutions
If I'm right you're wanting to send the string via a CLI as an argument, and have it generate the appropriate resolution?
What you're doing is "right" at first glance, but you could also just have it as normal integers on the LHS. They have to be translated, so there isn't much benefit to using them, when you consider how much of a PITA CLI's are.

Also, round((ppqn*4) /2 is exactly the same as round((ppqn*2)

For reference, this is taken from Melodramatik, which is in C++ of course, but you should be able to see the general idea:

Code: Select all

	switch (TimeDivisor)
	{
	case 0: fTimeDivisorPatterns[0] = 32; break; // 8/1
	case 1: fTimeDivisorPatterns[0] = 16; break; // 4/1
	case 2: fTimeDivisorPatterns[0] = 12; break; // 3/1
	case 3: fTimeDivisorPatterns[0] = 8; break; // 2/1
	case 4: fTimeDivisorPatterns[0] = 7; break; // 7/4
	case 5: fTimeDivisorPatterns[0] = 6; break; // 6/4
	case 6: fTimeDivisorPatterns[0] = 5; break; // 5/4
	case 7: fTimeDivisorPatterns[0] = 4; break; // 1/1
	case 8: fTimeDivisorPatterns[0] = 3; break; // 3/4
	case 9: fTimeDivisorPatterns[0] = 2; break; // 1/2
	case 10: fTimeDivisorPatterns[0] = 1.5; break; // 3/8
	case 11: fTimeDivisorPatterns[0] = 1; break; // 1/4 - At this size the step is 15360 PPQ
	case 12: fTimeDivisorPatterns[0] = 0.75; break; // 3/16
	case 13: fTimeDivisorPatterns[0] = 0.5; break; // 1/8
	case 14: fTimeDivisorPatterns[0] = 0.375; break; // 3/32 - 1/16 * 1.5 = 0.375
	case 15: fTimeDivisorPatterns[0] = 0.333333; break; // 1/8T - 0.333333r
	case 16: fTimeDivisorPatterns[0] = 0.25; break; // 1/16
	case 17: fTimeDivisorPatterns[0] = 0.166666; break; // 1/16T - 0.166666r
	case 18: fTimeDivisorPatterns[0] = 0.125; break; // 1/32
	case 19: fTimeDivisorPatterns[0] = 0.083333; break; // 1/32T - 0.083333r
	case 20: fTimeDivisorPatterns[0] = 0.0625; break; // 1/64
	case 21: fTimeDivisorPatterns[0] = 0.03125; break; // 1/128
	default: ASSERT(7 < 0);
		break;
	}


Cheers,
Brian

User avatar
Billy+
Posts: 4202
Joined: 09 Dec 2016

Post 17 Oct 2023

Enlightenspeed wrote:
16 Oct 2023

If I'm right you're wanting to send the string via a CLI as an argument, and have it generate the appropriate resolution?
What you're doing is "right" at first glance, but you could also just have it as normal integers on the LHS. They have to be translated, so there isn't much benefit to using them, when you consider how much of a PITA CLI's are.

Also, round((ppqn*4) /2 is exactly the same as round((ppqn*2)

Cheers,
Brian
thanks for reply and conformation. I have had a quick look at your code and it's interesting - but very random from what I could understand, but can't wait to see more as and when you release. especially as you made me realize I need to amend other values in the xml ;)

as for my code, I'm not using to many CLI parameters and the code is ultimately a testing script to manipulate and calculate values of interest. most of it is completely unnecessary and no doubt implemented wrong or at least could be done better. what I'm currently trying to solve is determining the best resolution / quantization value to use based on the gap between notes and also the shortest note in a given midi file as this will determine the "best" repatch resolution. however I still don't see any real probability of getting working code to take 3rd party midi and convert reasonable chunks into repatch files :- there just to dirty.

I am making progress on my sample data set however, and currently the output is looking a bit more promising, as an example :-

Code: Select all

PS C:\billy> python .\processmidi.py .\temp\PM-Templates\W-128.mid

PPQN 15360
BPM 120.0 
Track[0] Track Name[ID8 1] 
Durations to resolutions 
{'1/1': 61440, '1/2': 30720, '1/4': 15360, '1/8': 7680, '1/16': 3840, '1/16t': 2560, '1/32': 1920, '1/32t': 1280, '1/64': 960, '1/64t': 640, '1/128': 480}
Resolutions 8 : Min 1/128 : Max 1/1
extended note data:
{'pos': 1, 'pitch': 60, 'velocity': 100, 'start': 0, 'end': 61440, 'duration': '61440', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/1'}
{'pos': 2, 'pitch': 60, 'velocity': 100, 'start': 61440, 'end': 92160, 'duration': '30720', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/2'}
{'pos': 3, 'pitch': 60, 'velocity': 100, 'start': 92160, 'end': 107520, 'duration': '15360', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/4'}
{'pos': 4, 'pitch': 60, 'velocity': 100, 'start': 107520, 'end': 115200, 'duration': '7680', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/8'}
{'pos': 5, 'pitch': 60, 'velocity': 100, 'start': 115200, 'end': 119040, 'duration': '3840', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/16'}
{'pos': 6, 'pitch': 60, 'velocity': 100, 'start': 119040, 'end': 120960, 'duration': '1920', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/32'}
{'pos': 7, 'pitch': 60, 'velocity': 100, 'start': 120960, 'end': 121920, 'duration': '960', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/64'}
{'pos': 8, 'pitch': 60, 'velocity': 100, 'start': 121920, 'end': 122400, 'duration': '480', 'timetonext': '0', 'remainder': '1 0', 'notation': '1/128'}
PS C:\billy> 



I'm detecting notes that are complete divisions,
determining the time between notes,
that largest note duration and the smallest resolution value.

most of the output is simple to manipulate and although I've hit a snag with one of the dictionary's I can see a possible solution to this and just need to rewrite some code to work with the change.

there is still plenty to figure out just for my good midi files and I'm almost certain that 3rd party files are not going to happen.

either way if I can convert my dictionary file into midi files (which looks to be working) my data set makes it worth trying a bit harder to get the repatcher working :lol:

Code: Select all

crunch_win.exe 16 16 hmsr -d 1 -o rhythmlist.txt -u

PS E:\genmid> .\crunch_win.exe 16 16 hmsr -d 1 -o rhythmlistdup1.txt -u
Disabling printpercentage thread.  NOTE: MUST be last option

Crunch will now generate the following amount of data: 975725676 bytes
930 MB
0 GB
0 TB
0 PB
Crunch will now generate the following number of lines: 57395628 
each line is 1 midi file :oops:

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 20 Oct 2023

Hi folks,

Done enough with this now to be happy to call it a V1 :)
Pat_The_Mutt - RT release 1.0.zip
Added CV, accents, tie flags, and two cool new modes: Doubling, and Progression Doubling.
Doubling will double the pattern horizontally, so that in 32 steps the 2nd 16 is a repeat of the first 16. This works really, really well in Pat_Mutt as you get mutations where the first and last 4 bars are then very closely related, but subtly different.
Progression doubling does something quite different, in that it adds a second layer of the same events, but out of phase by half the cycle, so the pattern is doubled like above, but there is extra events in the 2nd half which harmonise the first half. Again this is really cool and I love the results.

Enjoy!
Brian

P.s. I have a few more tricks up my sleeve regarding this, but they may never see the light of day, as I have a lot of stuff to do elsewhere. I've noted what these are into my version of the Python file, so it's something for a rainy day.
You do not have the required permissions to view the files attached to this post.

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 20 Oct 2023

So, if I'm right, you have a large "dictionary" of MIDI files that you want to make available for RE patch formats?
Billy+ wrote:
17 Oct 2023
what I'm currently trying to solve is determining the best resolution / quantization value to use based on the gap between notes and also the shortest note in a given midi file as this will determine the "best" repatch resolution. however I still don't see any real probability of getting working code to take 3rd party midi and convert reasonable chunks into repatch files :- there just to dirty.
Not sure why you would do this, the 3rd party MIDI will be usually working in a better resolution than any pattern sequencer type tool.
Billy+ wrote:
17 Oct 2023
Crunch will now generate the following number of lines: 57395628 ...
...each line is 1 midi file :oops:
I can give a "near guarantee" here, that from this day until they put you in a coffin (hopefully because you have passed away, and not because you're family just kinda like putting living people in coffins)... that you would never use more than about 100,000 of that set of 57 million - that's going on a basis of using 10 a day over a period of 30 years, during which times other techs would be constantly improving; consider where music tech was in 1993. Just as likely you won't breach the 100 mark. So, for most people that's 100 useful vs 57395528 wasted.

I tend to find that huge libraries like this provide more distraction than value.

I would encourage you to continue for the sake of improving your coding, it's a cool exercise in that regard, but I think you need to be realistic about the results, which will most likely be unusable; Once you start quantizing natural grooves to hard 1/16ths and 1/32nd's you lose a massive amount of the musical value.

Give me a shout for questions on specifics and I'll see what I can do.

Cheers,
Brian

User avatar
Enlightenspeed
RE Developer
Posts: 1107
Joined: 03 Jan 2019

Post 29 Oct 2023

Hi Billy,

I was curious about this, so did some digging around for my own amusement. Turns out MIDI is an uglier beast than I originally imagined! I've written some code for reading it now (without using any Python imports) but in order to help you with your cause, and to turn it into repatch files, could you give me a small subset of the dictionary you are using?

I would only need about 20 entries worth, including the original header if it has one.

I'm going on hols to the sunshine in less than a week now, so I'll be unlikely to get much done with it until I'm back, thus a little patience may be required.

Cheers,
Brian

  • Information
  • Who is online

    Users browsing this forum: No registered users and 0 guests