pandas

    Provide efficient data manipulation and analysis

    pandas
    polars
    dataframe
    data cleaning
    data manipulation
    Download zip
    Version
    0.8.0
    License
    Apache-2.0
    Last updated
    14 hours ago
    Downloads
    113

    #Pandas In Moonbit

    mooncakes.io link: https://mooncakes.io/docs/smallbearrr/pandas

    moonbit-community link: https://github.com/moonbit-community/pandas

    #Introduction

    This is a data processing library written in Moonbit. It provides a DataFrame data structure for efficient data manipulation and analysis.

    #Features

    • DataFrame and Series data structures for handling tabular data
    • Support for various data types including integers, floats, strings, and booleans
    • Basic data manipulation operations such as filtering, sorting, and aggregating
    • Support for reading and writing CSV files
    • Support for handling missing values and null data
    • Support for mathematical operations on Series and DataFrames

    #Installation

    To install the package, run the following command in the terminal:
    moon add smallbearrr/pandas

    #Dependencies

    This package depends on the following libraries:
    • "moonbitlang/x/fs": A file system I/O library.
    • "moonbit-community/NyaCSV": A CSV parsing library.

    To install these dependencies, run the following command:
    moon add moonbitlang/x
    moon add moonbit-community/NyaCSV

    #Usage

    The snippets below are meant to run inside a test block or a function that can raise errors. For brevity the types are written without the @pandas. prefix.

    #DataFrame

    create a DataFrame
    let col1 = Series::new("A", SeriesInput::Int32([1, 2, 3]))
    let col2 = Series::new("B", SeriesInput::Float32([1.5, 2.5, 3.5]))
    let col3 = Series::new("C", SeriesInput::Int32_Nullable([Some(1), None, Some(3)]))
    let df = DataFrame::new([col1, col2, col3])

    Display the first few(min(size, 5)) rows of the DataFrame
    df.head()
    A B C 0 1 1.5 1 1 2 2.5 null 2 3 3.5 3

    DataFrame already traits the Show, it can print the DataFrame directly:
    println(df)
    A B C 0 1 1.5 1 1 2 2.5 null 2 3 3.5 3

    Add a new column
    let new_col = Series::new("D", SeriesInput::Bool([true, false, true]))
    df.add_column(new_col)
    println(df)
    A B C D 0 1 1.5 1 true 1 2 2.5 null false 2 3 3.5 3 true

    Drop a column
    df.drop_column("C")
    println(df)
    A B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true

    Select a column
    let df_col = df.column("A")
    println(df_col.data())
    Int([1, 2, 3])

    Select specific columns
    let df_cols = df.select_columns(["A", "B"])
    println(df_cols)
    A B 0 1 1.5 1 2 2.5 2 3 3.5

    Add a new row
    df.add_row([SeriesValue::Int(7), SeriesValue::Float(7.5), SeriesValue::Bool(true)])
    println(df)
    A B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true 3 7 7.5 true

    Drop a row (drop_row currently only works when every column was created from a nullable input such as Int32_Nullable, so this example uses a separate DataFrame)
    let df_nullable = DataFrame::new([
    Series::new("A", SeriesInput::Int32_Nullable([Some(1), None, Some(3)])),
    Series::new("B", SeriesInput::Float32_Nullable([Some(1.5), Some(2.5), None])),
    ])
    df_nullable.drop_row(0)
    println(df_nullable)
    A B 0 null 2.5 1 3 null

    Select rows by range or by specific indices:
    let row_selected_range = df.select_rows(range=(1, 3))
    let row_selected_indices = df.select_rows(indices=[0, 2])
    println(row_selected_range)
    println(row_selected_indices)
    A B D 0 2 2.5 false 1 3 3.5 true A B D 0 1 1.5 true 1 3 3.5 true

    Filter rows based on a condition applied to a specific column:
    let filtered = df.filter("A", fn(x) { x < SeriesValue::Int(3) })
    println(filtered)
    A B D 0 1 1.5 true 1 2 2.5 false

    Sort the DataFrame by a specified column in ascending or descending order:
    df.sort("A", descending=true)
    println(df)
    df.sort("B")
    println(df)
    A B D 0 7 7.5 true 1 3 3.5 true 2 2 2.5 false 3 1 1.5 true A B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true 3 7 7.5 true

    Rename a column (rename does not update the column lookup of the DataFrame, so do the renaming last)
    df.rename("A", "A1")
    println(df)
    A1 B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true 3 7 7.5 true

    Convert Between DataFrame and @NyaCSV.CSV:
    let csv = df.to_NyaCSV()
    let df_from_csv = DataFrame::from_NyaCSV(csv)
    println(df_from_csv)
    A1 B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true 3 7 7.5 true

    Write the DataFrame to a CSV file (write_csv only writes to a file that already exists):
    df.write_csv("./output.csv")

    Read a CSV file:
    let df_read = DataFrame::read_csv("./output.csv")
    println(df_read)
    A1 B D 0 1 1.5 true 1 2 2.5 false 2 3 3.5 true 3 7 7.5 true

    #Series

    Create a Series
    let series_int = Series::new("IntColumn", SeriesInput::Int32([1, 2, 3]))
    let series_float = Series::new("FloatColumn", SeriesInput::Float32([1.1, 2.2, 3.3]))
    let series_bool = Series::new("BoolColumn", SeriesInput::Bool([true, false, true]))
    let series_str = Series::new("StrColumn", SeriesInput::Str(["a", "b", "c"]))
    println(series_int.data())
    println(series_float.data())
    println(series_bool.data())
    println(series_str.data())
    Int([1, 2, 3]) Float([1.1, 2.2, 3.3]) Bool([true, false, true]) Str([a, b, c])

    Erase element (erase currently only works on a Series created from a nullable input such as Int32_Nullable)
    let series_null = Series::new("NullColumn", SeriesInput::Int32_Nullable([Some(1), None, Some(3)]))
    series_null.erase(1)
    println(series_null.data())
    Int([1, 3])

    Sort a Series in place, and get the indices that would sort it
    let series = Series::new("S", SeriesInput::Int32([3, 1, 2]))
    let indices = series.argsort()
    series.sort()
    println(indices)
    println(series.data())
    [1, 2, 0] Int([1, 2, 3])

    Add two Series
    let series1 = Series::new("A", SeriesInput::Int32([1, 2, 3]))
    let series2 = Series::new("B", SeriesInput::Float32([1.5, 2.0, 3.5]))
    let add = series1 + series2
    println(add.data())
    Float([2.5, 4, 6.5])

    Operation of add, subtract, multiply and divide
    let add_op = series1 + series2
    let sub_op = series1 - series2
    let mul_op = series1 * series2
    let div_op = series1 / series2
    println(add_op.data())
    println(sub_op.data())
    println(mul_op.data())
    println(div_op.data())
    Float([2.5, 4, 6.5]) Float([0.5, 0, 0.5]) Float([1.5, 4, 10.5]) Float([1.5, 1, 1.1666666])

    Extend a Series in-place with another Series of the same type
    let a = Series::new("t", SeriesInput::Int32([1, 2]))
    let b = Series::new("t", SeriesInput::Int32([3, 4]))
    a.extend_from(b)
    println(a.data())
    Int([1, 2, 3, 4])

    #Feature

    #DataFrame Methods

    MethodDescription
    newCreate a new DataFrame
    shapeGet the shape of the DataFrame
    dataGet the data of the DataFrame
    headDisplay the first few rows of the DataFrame
    add_columnAdd a new column to the DataFrame
    drop_columnDrop a column from the DataFrame
    renameRename a column in the DataFrame
    columnSelect a column from the DataFrame
    select_columnsSelect specific columns from the DataFrame
    drop_rowDrop a row from the DataFrame
    add_rowAdd a new row to the DataFrame
    select_rowsSelect specific rows from the DataFrame
    filterFilter rows in the DataFrame based on a condition
    sortSort the DataFrame by a specified column
    vstackVertically stack two DataFrames
    hstackHorizontally stack two DataFrames
    clearClear all data in the DataFrame
    cloneCreate a deep copy of the DataFrame
    itemRetrieve a single item from the DataFrame
    limitCreate a new DataFrame containing only the first N rows
    tailCreate a new DataFrame containing only the last N rows
    sliceCreate a new DataFrame containing a slice of rows
    to_seriesConvert a column of the DataFrame to a Series
    uniqueRemove duplicate rows from the DataFrame
    replace_columnReplace a column in the DataFrame
    reverseReverse the order of rows in the DataFrame
    get_column_indexGet the index of a column by its name
    with_row_indexAdd a row index column to the DataFrame
    transposeTranspose the DataFrame over the diagonal
    countCount the number of rows and columns in the DataFrame
    maxGet the maximum value of each column in the DataFrame
    max_horizontalGet the maximum value of each row in the DataFrame
    minGet the minimum value of each column in the DataFrame
    min_horizontalGet the minimum value of each row in the DataFrame
    sumCalculate the sum of each column in the DataFrame
    sum_horizontalCalculate the sum of each row in the DataFrame
    dtypesGet the data types of each column in the DataFrame
    heightGet the number of rows in the DataFrame
    widthGet the number of columns in the DataFrame
    productCalculate the product of each column in the DataFrame
    schemaGet the schema of the DataFrame
    is_emptyReturn whether the DataFrame is empty
    fill_nullFill null values using the specified value or strategy
    null_countReturn the number of null values in the DataFrame
    castCast DataFrame column(s) to the specified dtype(s)
    to_dictConvert the DataFrame to a dictionary
    to_init_reprConvert DataFrame to instantiable string representation
    drop_nullsDrop all rows that contain null values
    gather_everyTake every nth row in the DataFrame and return as a new DataFrame
    drop_in_placeDrop a single column in-place and return the dropped column
    read_csvRead a CSV file and convert it into a DataFrame
    from_NyaCSVConvert a NyaCSV.CSV object into a DataFrame
    to_NyaCSVConvert the DataFrame to a NyaCSV.CSV object
    write_csvWrite the DataFrame to a CSV file

    #Series Methods

    MethodDescription
    newCreate a new Series
    dataGet the data of the Series
    eraseErase an element from the Series
    sortSort the Series in place
    +Add two Series
    -Subtract two Series
    *Multiply two Series
    /Divide two Series
    meanCalculate the mean of the Series
    varianceCalculate the variance of the Series
    argsortReturn the indices that would sort the Series
    copyCreate a copy of the Series
    countCount the number of elements in the Series
    get_typeGet the type of the Series
    maxGet the maximum value of the Series
    mergeMerge two Series
    minGet the minimum value of the Series
    nameGet the name of the Series
    reverseReverse the order of elements in the Series
    sumCalculate the sum of the Series
    absReturn a new Series with the absolute value of each element
    acosReturn a new Series with the arccos of each element
    allReturn whether all non-null elements in the Series are true
    anyReturn whether any non-null element in the Series is true
    appendAppend another Series to the current one and return a new Series
    clearRemove all elements from the current Series
    concatConcatenate multiple Series and return the result
    cosReturn a new Series with the cosine of each element
    extend_fromExtend the current Series in-place with another Series
    has_nullCheck if the current Series contains any null values
    is_emptyCheck if the current Series is empty
    is_not_nullReturn a boolean Series indicating non-null elements
    is_nullReturn a boolean Series indicating null elements
    is_sortedCheck if the current Series is sorted
    lengthReturn the length of the current Series
    logReturn a new Series with the logarithm of each element
    log10Return a new Series with the base-10 log of each element
    lower_boundReturn a new Series with each element replaced by the lower bound
    n_uniqueReturn the number of unique elements in the Series
    not_Return the logical NOT of a boolean Series
    null_countReturn the count of null values in the Series
    op_getReturn the element at a given index in the Series
    sinReturn a new Series with the sine of each element
    tanReturn a new Series with the tangent of each element
    unique_countsReturn a new Series showing the count of each unique value
    upper_boundReturn a new Series with each element replaced by the upper bound
    sliceReturn a new Series containing elements in the specified range
    swapSwap two elements in the Series
    get_argsort_indicesGet the indices that would sort the Series
    move_nulls_frontMove null values to the front of the Series
    move_nulls_backMove null values to the back of the Series

    #Contributing

    Issues and pull requests are welcome. Please make sure to run all tests before submitting.

    #Contact

    If you have questions, feel free to open an issue in the repository. For other inquiries, contact me via email: [luyaolyu1@gmail.com].

    #License

    This project is licensed under the Apache-2.0 License. See the LICENSE file for details.

    #Changelog

    #0.8.0

    • Adapted to moon 0.1.20261119 and the new moon.mod / moon.pkg format.
    • Dependencies are now moonbit-community/NyaCSV@0.3.4 and moonbitlang/x@0.5.8.
    • Series::extend is renamed to Series::extend_from, because extend is now a keyword.
    • The old top-level function style (e.g. @pandas.limit(df, n)) is no longer available; use method calls such as df.limit(n).
    • Show derived by the new standard library prints strings without quotes; use Debug for the old format.