Kotlin DataFrame: typesafe in-memory structured data processing for JVM

Kotlin DataFrame aims to reconcile Kotlin's static typing with the dynamic nature of data by utilizing both the full power of the Kotlin language and the opportunities provided by intermittent code execution in Jupyter notebooks and REPL.

• Hierarchical — represents hierarchical data structures, such as JSON or a tree of JVM objects.
• Functional — data processing pipeline is organized in a chain of DataFrame transformation operations. Every operation returns a new instance of DataFrame reusing underlying storage wherever it's possible.
• Readable — data transformation operations are defined in DSL close to natural language.
• Practical — provides simple solutions for common problems and the ability to perform complex tasks.
• Minimalistic — simple, yet powerful data model of three column kinds.
• Interoperable — convertable with Kotlin data classes and collections.
• Generic — can store objects of any type, not only numbers or strings.
• Typesafe — on-the-fly generation of extension properties for type safe data access with Kotlin-style care for null safety.
• Polymorphic — type compatibility derives from column schema compatibility. You can define a function that requires a special subset of columns in a dataframe but doesn't care about other columns.

Integrates with Kotlin kernel for Jupyter. Inspired by krangl, Kotlin Collections and pandas

Documentation

Explore documentation for details.

You could find the following articles there:

• Get started with Kotlin DataFrame
• Working with Data Schemas
• Setup compiler plugin in Gradle project
• Full list of all supported operations
  • Reading from SQL databases
  • Reading/writing from/to different file formats like JSON, CSV, Apache Arrow
  • Joining dataframes
  • GroupBy operation
• Rendering to HTML

What's new

1.0.0-Beta2: Release notes

Check out this notebook with new features in v0.15.

Setup

implementation("org.jetbrains.kotlinx:dataframe:1.0.0-Beta2")

Check out the custom setup page if you don't need some of the formats as dependencies, for Groovy, and for configurations specific to Android projects.

Code example

import org.jetbrains.kotlinx.dataframe.*
import org.jetbrains.kotlinx.dataframe.api.*
import org.jetbrains.kotlinx.dataframe.io.*

val df = DataFrame.read("https://raw.githubusercontent.com/Kotlin/dataframe/master/data/jetbrains_repositories.csv")
df["full_name"][0] // Indexing https://kotlin.github.io/dataframe/access.html

df.filter { "stargazers_count"<Int>() > 50 }.print() 

Getting started in Kotlin Notebook

Follow this guide

Data model

• DataFrame is a list of columns with equal sizes and distinct names.
• DataColumn is a named list of values. Can be one of three kinds:
  • ValueColumn — contains data
  • ColumnGroup — contains columns
  • FrameColumn — contains dataframes

Explore more examples here.

Kotlin, Kotlin Jupyter, Arrow, and JDK versions

This table shows the mapping between main library component versions and minimum supported Java versions.

Kotlin DataFrame Version	Minimum Java Version	Kotlin Version	Kotlin Jupyter Version	Apache Arrow version
0.10.0	8	1.8.20	0.11.0-358	11.0.0
0.10.1	8	1.8.20	0.11.0-358	11.0.0
0.11.0	8	1.8.20	0.11.0-358	11.0.0
0.11.1	8	1.8.20	0.11.0-358	11.0.0
0.12.0	8	1.9.0	0.11.0-358	11.0.0
0.12.1	8	1.9.0	0.11.0-358	11.0.0
0.13.1	8	1.9.22	0.12.0-139	15.0.0
0.14.1	8	2.0.20	0.12.0-139	17.0.0
0.15.0	8	2.0.20	0.12.0-139	18.1.0
1.0.0-Beta2	8 / 11	2.0.20	0.12.0-383	18.1.0

Code of Conduct

This project and the corresponding community are governed by the JetBrains Open Source and Community Code of Conduct. Please make sure you read it.

License

Kotlin DataFrame is licensed under the Apache 2.0 License.