[Proposal] Add a mechanism for generating a dataset not based on a Gymnasium etc source but from a plain time series data

[jamartinh]

Proposal

Add a mechanism for generating a dataset not based on a Gymnasium etc source but from a plan time series data.

Motivation

In much scenarios the data for OfflineRL is coming from a plain dataset, e.g. pandas, numpy etc. This makes more or less tedious or difficult to create a minari dataset from it. So how about creating a clear way of having an clear interface for writing a Minari dataset from a time series data stored in pandas, numpy or other plain dataset format without having to have a Gymnasium environment ?

Alternatives

• Write manually an hdf5 or parquet with the minari format data that will require using knowledge of the internals of minari.
• Create a fake environment that iterates over the dataset to generate the minari dataset

[younik]

Hi @jamartinh, we aim to support this indeed.

We should already support it with EpisodeBuffer and minari.create_dataset_from_buffers.
The only remaining dependency on Gymnasium in that case is in defining the action and observation spaces using Gymnasium. Is this a limitation for your use case?

[jamartinh]

Thanks I have seen this function right now. It should work, but can the obs and action spaces be optional? Or is it a hard requirement for Minari datasets? Or perhaps a Space.Unbounded? El jue, 20 feb 2025, 14:17, Omar Younis ***@***.***> escribió:

…

Hi @jamartinh <https://github.com/jamartinh>, we aim to support this indeed. We should already support it with EpisodeBuffer and minari.create_dataset_from_buffers <https://minari.farama.org/api/minari_functions/#create-minari-dataset>. The only remaining dependency on Gymnasium in that case is in defining the action and observation spaces using Gymnasium. Is this a limitation for your use case? — Reply to this email directly, view it on GitHub <#276 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA3NNMYP3RXYYVV4ZKIJCCD2QXIW7AVCNFSM6AAAAABXQUD24GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZRGQ3DSNJVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: younik]*younik* left a comment (Farama-Foundation/Minari#276) <#276 (comment)> Hi @jamartinh <https://github.com/jamartinh>, we aim to support this indeed. We should already support it with EpisodeBuffer and minari.create_dataset_from_buffers <https://minari.farama.org/api/minari_functions/#create-minari-dataset>. The only remaining dependency on Gymnasium in that case is in defining the action and observation spaces using Gymnasium. Is this a limitation for your use case? — Reply to this email directly, view it on GitHub <#276 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA3NNMYP3RXYYVV4ZKIJCCD2QXIW7AVCNFSM6AAAAABXQUD24GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZRGQ3DSNJVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[younik]

Thanks I have seen this function right now. It should work, but can the obs
and action spaces be optional? Or is it a hard requirement for Minari
datasets? Or perhaps a Space.Unbounded?

El jue, 20 feb 2025, 14:17, Omar Younis @.***> escribió:
…

At the moment it cannot be optional, but you can set the space to be as generic as possible, e.g. a Box without bounds. However, we expect the shape and types of arrays to be the same across the episode, as well as the structure in the case of a nested array.