Wire-free Miniscope v4

The wire-free Miniscope v4 is a battery powered, data logging Miniscope based around the original wired Miniscope v4 project. All code, CAD, and PCB files can be found at https://github.com/Aharoni-Lab/Miniscope-v4-Wire-Free

Specifications

• Weight:
• Size:
• Field of View:
• Recording Resolution:
• Recording Length:
• Record Triggering:

Overview of System

Workflow of Recording

Setting the conda environment

First we need to set a conda environment with the Python packages needed for this project: opencv, matplotlib and numpy. If you don't have Anaconda installed in your computer you can download the installer from the official Anaconda website. You can manually install the packages using pip or using the included requirements.txt file. To do so, clone the Wire-Free repository and open an Anaconda prompt terminal in the folder Miniscope-v4-Wire-Free-MCU-Firmware.

conda create --name YOUR_ENV_NAME --file requirements.txt

where YOUR_ENV_NAME is the name you choose for your environment. As an example, I will choose the name WFUtils. Hit enter to run the line and wait until it finishes solving the environment

Anaconda prompt to create a conda environment using the requirements.txt file and with a WFUtils name.

Once it finishes it will show the following. Press Y to confirm

Dialog that asks if we want to create the environment and install the packages.

If the installation was successful, the following neding dialog will show up

Dialog showing that the environment was created successfully

The code shown in the image above is needed to activate the environment. You may now close Anaconda Prompt.

Installing HxD

HxD is a software that can be used to check the contents of any physical drive connected to a computer. It can be used here to check the contents of the SD card that will store the configuration parameters and imaging data. You can download this software here.

Setting the recording parameters in the SD Card

First run anaconda prompt in admin mode

How to run anaconda prompt in administrator mode.

To move to your path easier, copy paste the path and run the cd command

cd YOUR_PATH

Command to change directory.

Once in the folder, run the following line to open Jupyter

jupyter notebook

Landing page once the command jupyter notebook is run.

Open Set recording parameters. Before running anything lets check the physical drive number for the SD card. We can do this running Get-PhysicalDisk on a Windows Powershell. Before connecting the SD Card we can see the physical drives connected to the computer. In my case, I have two SSD drives. The enumeration starts at zero.

Screenshot showing the output of Get-PhysicalDisk before plugging in the SD card.

Image showing the updated list of physical drives connected to the computer after plugging in the SD card.

The notebook has a comments explaining what each cell is doing, however certain parameters need to be explained. To start lets make sure that we are effectively working with the SD card. In the following line the number N should be changed to the actual number of the physical drive. From the screenshot above we can see that the SD card the physical drive 2

driveName = r"\\.\PhysicalDriveN"

Then we run this cell to erase all the previous data saved in the SD card. Make sure that your data has already been saved in your computer before running this cells. After running this cell all data is permanently erased

# Code to erase any previously saved data (if any) in the data sector of the microSD card

f.seek(headerSector * sectorSize, 0)
zeros = np.zeros(sectorSize, dtype=np.uint8)
binaryZeros = bytearray(zeros)
N = 1000000
for i in range(N):
    f.write(binaryZeros)

Now we set the parameters in the header sector. It's important to remember where the data sectors are

# SD Card sector information
headerSector =          1022 # Holds user settings to configure Miniscope and recording
configSector =          1023 # Holds final settings of the actual recording
dataStartSector =       1024 # Recording data starts here
sectorSize =            512

WRITE_KEY0 =				0x0D7CBA17
WRITE_KEY1 =				0x0D7CBA17
WRITE_KEY2 =				0x0D7CBA17
WRITE_KEY3 =				0x0D7CBA17

# SD Card Header Sector positions
HEADER_GAIN_POS =				4
HEADER_LED_POS =				5
HEADER_EWL_POS =				6
HEADER_RECORD_LENGTH_POS =  	7
HEADER_FRAME_RATE = 			8
HEADER_DELAY_START_POS =		9
HEADER_BATT_CUTOFF_POS =		10

Here we set the values for the Miniscope. As an example, we will set the gain of the image sensor at 1, the LED at 2, the electrowetting lens at 150 and 20 FPS. We will record for 20 seconds with a 10 second delay since we trigger the recording, and if the voltage drops less than 3300 mV (3.3V) the recording will stop automatically even if it didn't record for the time set before.

# Set the Miniscope parameters
# We have to multiply the index by 4 so they can be 32 bit long (4*byte = 32 bits)

gain = 1                        # Gain 1= 1, Gain 2= 2, Gain 3= 4
led = 2                         # 0 to 100 (0 = LED off)
ewl_pos = 150                   # EWL range= 1 to 255 (0 = EWL off)
recording_length = 15         # Recording length (in seconds)
frame_rate = 20                 # In FPS
battery_cutoff = 3300           # Battery level (millivolts)
delay_start = 10                # In seconds

Once everything is verified, run all the cells. We can check that the parameters have been saved in two different ways. Within the notebook, running the following cell will output the saved parameters

f.seek(headerSector * sectorSize, 0)  # Move to correct sector
headerSectorData = np.frombuffer(f.read(sectorSize), dtype=np.uint8)
headerSectorData

For the settings above, we get the following output. For parameters that have a value less than 255 we can read them directly from their memory position. For example, the setting of the LED with value 2 can be found in the at the position (4*HEADER_LED_POS)+1 = 21. The position in the header can be found at the beginning of the notebook.

array([  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   1,   0,   0,   0,   2,   0,   0,   0, 150,   0,
         0,   0,  15,   0,   0,   0,  20,   0,   0,   0,  10,   0,   0,
         0, 228,  12,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0], dtype=uint8)

Another way of checking the same is using HxD. In order to check the contents of the SD Card we need to open it in admin mode. The installer will give two options: 32 or 64 bits. If unsure choose HxD64 as it's highly likely that your computer architecture is 64 bits.

To open the disk, go here

How to open the SD card on HxD.

The following dialog will pop

If unsure as which disk is, you have two options. First is to sort by disk size. The Wire-Free ships with a 32 GB SD card, so choose the drive with size similar to that. The other option is to check the hardware name that we got on GetPhysicalDisk before. In this case the name is NORELSYS 1081CS0. To check the content we first need to go to the sector 1022, where the header data is. The sector locations can be found at the beginning of the notebook

To check the contents

MCU Firmware Walkthrough

The microcontroller (MCU) used in the Wire-Free Miniscope v4 is an ATSAMD51. It sits between the Python 480 CMOS image sensor and the micro SD Card and handles all on-board control and operation of the Miniscope. The firmware uses a combination of custom written driver and ones imported using ATMEL SMART.

Initialization

• atmel_start_init()
  • This function calls init_mcu() which sets up clocks, configs and enable DMA, and enables/disables cache.
  • This function also sets up the initial configurations mainly defined by the ATMEL SMART interface. This includes setting GPIO direction and mode and initalizing the ADC, external interrupts, PPC, USART, PWM, SYSTick, and Timers.
  • This function also does some minor stuff to initalize the sd_mmc stack.
• Next we do some additional manual configuration of the PWM mode, enabling the 3.3V external regulator, and enabling the ADC.
• Due to pin limitations when using both the PCC module and SD Card module, we don't have access to directly using the SERCOM I2C functionality of this particular MCU. This means we need to implement I2C ourselves using a "Big Back" approach. We set this all up in I2C_BB_init() and all of the I2C Big Bang functionality is done in i2c_bb.c.
• Next we setup timers that let us keep track of the time in milliseconds, TIMER_0_taks1, and check the Lipo battery voltage every once-in-a-while, TIMER_0_task2.
• Next we construct our DMA/Buffer linked lists in linkedListInit(). More information on this can be found below.
• Next we wait until we detect an SD Card in mounted on the PCB, read the header memory block (block 1022) using loadSDCardHeader(), and configure the SDHC module to run using ADMA.
• Next we setup the Python 480 by toggling its reset pin, sending it initial register values using python480Init(), and then configuring its registers for subsampling, gain, FPS, etc.
• Finally, we write some initial configuration information into SD card memory block 1023 and then set deviceState = DEVICE_STATE_START_RECORDING to trigger the event to begin recording imaging data.

Buffers

The MCU firmware creates a circular buffer to fill and hold pixel data. This set of buffers should each have a size a multiple of 512 bytes so that each can be nicely written into an SD Card whose memory blocks are 512 bytes in size. Outside of the constraint that each buffer should be some multiple of 512 bytes there are no other constraints in terms of number of buffers or buffer size relative to image sensor frame size.

The circular buffers are defined globally within main.c :

COMPILER_ALIGNED(4)
volatile uint32_t dataBuffer[NUM_BUFFERS][BUFFER_BLOCK_LENGTH * BLOCK_SIZE_IN_WORDS]; //Allocate memory for DMA image buffers

This set of buffers gets circularly iterated through using DMA linked lists as pixel data arrives from the image sensor. Once a buffer is full, or once the end of a frame arrives and we have a partially filled buffer, the buffer becomes available to be written to the SD Card using ADMA.

State Machine

// ---------- Device State Definitions -------
#define DEVICE_STATE_IDLE				1<<1
#define DEVICE_STATE_START_RECORDING	1<<2
#define DEVICE_STATE_RECORDING			1<<3
#define DEVICE_STATE_STOP_RECORDING		1<<4
#define DEVICE_STATE_CHARGING			1<<5
#define DEVICE_STATE_CONFIG_LOADED		1<<6
#define DEVICE_STATE_ERROR				1<<7
#define DEVICE_STATE_LOW_VOLTAGE		1<<8
#define DEVICE_STATE_START_RECORDING_WAITING	1<<9
#define DEVICE_STATE_SDCARD_WRITE_ERROR			1<<10
#define DEVICE_STATE_SDCARD_INIT_WRITE_ERROR	1<<11

• The device state initially starts off in DEVICE_STATE_IDLE.
• To begin recording, the device start needs to be set to DEVICE_STATE_START_RECORDING. This lets the MCU know that we need to prepare everything to start a new recording session.
• Once the MCU prepares itself, it moves its device state to DEVICE_STATE_START_RECORDING_WAITING. This state is managed within the frameValid_cb function and makes the MCU wait until the end of the last frame is received before enabling the PCC DMA so that data buffers can start being filled and counted when the start of the next frame begins to arrive. Once the MCU enables the PCC DMA, it will move the device state to DEVICE_STATE_RECORDING.
• Once the MCU moves into the device = DEVICE_STATE_RECORDING, management of detecting newly filled buffers and sending those over to the SD Card is all handed within the function void recording(). recording() is continuously called within the while loop in main().

Callback Functions

Callback functions handle specific events within the MCU that generate interrupts. Roughly speaking, when certain interrupts happen, the MCU will jump from wherever it is currently in its code, to the callback function linked to this interrupt event. Callback functions are denoted with a '_cb' at the end of their function name:

// callbacks
static void millisecondTimer_cb(const struct timer_task *const timer_task);
static void checkBattVoltage_cb(const struct timer_task *const timer_task);

static void battCharging_cb(void);
static void irReceive_cb(void);
static void pushButton_cb(void);
static void frameValid_cb(void);

Many of these callback functions are connected to a specific external interrupt (basically the input on an MCU pin changing a certain way) which is done near the top of main():

// Setup callbacks for external interrupts
	ext_irq_register(PIN_PB22, irReceive_cb);
	ext_irq_register(PIN_PB23, battCharging_cb);
	ext_irq_register(PIN_PB14, frameValid_cb);
	ext_irq_register(PIN_PA25, pushButton_cb);

The DMA callback function is pcc_dma_cb() and gets linked to the DMA interrupt in main() using camera_async_register_callback.

DMA

The MCU using Direct Memory Access (DMA) to stream pixel data into memory from the image sensor as well as write imaging data from memory into the SD Card. For pixel data coming from the image sensor, the MCU uses its Parallel Capture Controller (PCC) with DMA to handle an 8-bit parallel pixel bus. For writing imaging data into the SD Card, the MCU uses its ADMA to move raw data in the MCU memory into the SD Card storage blocks.

PCC DMA (Image Sensor -> MCU Memory)

Linked List DMA Descriptors

COMPILER_ALIGNED(16) // Taken from hpl_dmac.c but I think this could be '8' since descriptors need to be 64bit aligned from data sheet
volatile DmacDescriptor linkedList[NUM_BUFFERS];

PCC DMA Callback

Once a DMA transfer has completed, the pcc_dma_cb function gets called. This callback function will update the header portion of the last written into buffer using the setBufferHeader function and increment the bufferCount and frameBufferCount.

SD Card ADMA (MCU Memory -> SD Card)

Writing and reading from the SD Card takes advantage of ADMA. I can't quite remember what all the differences are here between ADMA and regular DMA but it took a while to get up and running. To get everything up and running in the firmware, I added two additional functions in sd_mmc.c:

• sd_mmc_err_t sd_mmc_write_with_ADMA(uint8_t slot, uint32_t start, uint32_t *descAdd, uint16_t nb_block)
• sd_mmc_err_t sd_mmc_wait_end_of_ADMA_write(bool abort)