Experimental data saving structure

Idea

In every action (grasping/placing) all the specified sensor data is recorded alongside with its timestamp. This information is saved separetly in *.npy files: (sensor_id)values.npy and (sensor_id)timestamps.npy. All this files are gathered in the same action folder named: /(action_type)(action_id). Which is stored with all the other actions in the main experiment folder named: /EXP(exp_id).

In addition, a *.csv file called experiment_summary.csv is created containing a summary of every action performed by the robot. This file contains information such as: the id of the action, the folder path to get the sensor values, the number of readings of every sensor, if the grasp was sucesful...

The main idea of using this structure is to be able to filter and select the action information of the actions that we want to study by only reading the experiment_summary.csv file. This will let us get to the information that we desire without having to load everything, which will be much faster.

Folder structure

• /EXP_(exp_id)
  • experiment_summary.csv
  • /grasping_(action_id1)
    • (sensor_id1)_values.npy
    • (sensor_id1)_timestamps.npy
    • (sensor_id2)_values.npy
    • (sensor_id2)_timestamps.npy
    • ...
  • /grasping_(action_id2)
    • sensor_id1_values.npy
    • sensor_id1_timestamps.npy
    • ...
  • ...
  • /placing_(action_idn)
  • ...

Experiment_summary.csv fields

For every grasp we can find the following information:

    'exp_id'
    'action_id'
    'object_id'
    'action'
    'tote_num'
    'frame_rate_ratio'
    'image_size'
    'path'
    'duration'
    'start_time' 
    'end_time'
    'success'

+ (for every sensor recorded, the number of readings that got stored)

Sensors recorded

Some of them won't be recorded because in that particular action they don't provide usefull information (eg: ws_0 when performing a grasp on tote 1)

                'ws_0': {'topic':'ws_stream{}'.format(0), 'msg_format':Float64},
                'ws_1': {'topic':'ws_stream{}'.format(1), 'msg_format':Float64},
                'ws_2': {'topic':'ws_stream{}'.format(2), 'msg_format':Float64},
                'gs_image': {'topic':'rpi/gelsight/raw_image', 'msg_format':Image},
                'gs_deflection': {'topic':'rpi/gelsight/deflection', 'msg_format':Int32},
                'gs_contactarea': {'topic':'rpi/gelsight/contactarea', 'msg_format':gelsight_contactarea},
                'hand_commands': {'topic':'/hand_commands', 'msg_format':JointState},
                'grasp_status': {'topic':'/grasp_status', 'msg_format':JointState},
                'joint_states':{'topic':'/joint_states', 'msg_format':JointState},
                'grasp_all_proposals': {'topic':'/grasp_all_proposals', 'msg_format':Float32MultiArray},
                'grasp_proposal': {'topic':'/grasp_proposal', 'msg_format':Float32MultiArray},
                'rgb_bin0': {'topic':'/arc_1/rgb_bin0', 'msg_format':Image},
                'rgb_bin1': {'topic':'/arc_1/rgb_bin1', 'msg_format':Image},
                'depth_bin0': {'topic':'/arc_1/depth_bin0', 'msg_format':Image},
                'depth_bin1': {'topic':'/arc_1/depth_bin1', 'msg_format':Image},
                'wsg_driver': {'topic':'/wsg_50_driver/status', 'msg_format':Status},
                'exp_comments': {'topic':'/exp_comments', 'msg_format':String},
                'impact_time': {'topic':'/impact_time', 'msg_format':String},
                'objectType': {'topic':'/objectType', 'msg_format':String}