Changelog

For getML Enterprise & Community editions

Overhaul and better integration of API documentation and web page:
- Switch from sphinx to mkdocs
- Restructuring of User Guide, multiple amendments to documentation
Introduce strict typing regiment for feature learning aggregations and loss functions
Clean up and maintenance of example notebooks, make them executable in Colab
More informative progress bar and status updates using rich
Completely reworked IO
- Improved reliability, speed and maintainability by capitalizing on PyArrow
Introduce reflect-cpp for parsing and de/serialization
Availability of getML Docker runtime as a service on Docker Hub, allowing for easy setup

Complete rework of the build pipeline (docker and linux native)
- Introduce CCache, conan, vcpkg
- User multi-stage docker builds leveraging buildx and buildkit
- Centralized VERSION
Ruff for linting and formatting
Hatch for python package management

Generalization of Placeholder.join's on argument
Improved timestamp handling
Slicing improvements
- Slicing of DataFrames returned wrong results: Remove short circuit for slices with upper bound
- Introduce set semantics for slicing of DataFrame (return empty collections instead of erroring)
Fix displaying of parameter lists with values that exceed the presentable width
Fix displaying of DataFrames with one row or less
Fix progress bar output on Google Colab

Accelerated feature learning through Fastboost
Improved modelling on huge datasets through ScaleGBMClassifier and ScaleGBMRegressor
Advanced trend aggregations using EWMATrend aggregations
Faster JSON parsing using YYJSON

Use websockets instead of polling
Size threshold for better visualization of feature code
Faster reading of memory-mapped data, relevant for all feature learners and predictors
Introduce CategoryTrimmer as preprocessor

Enhance data processing by introducing Spark (e.g. spark_sql) and Arrow (e.g. from_arrow())
Integrate Vcpkg for dependency management
Improve code transpilation for seasonal variables
Better control of predictor training and hyperparamter optimization through introduction of early stopping (e.g. in ScaleGBMClassifier)
Introduce TREND aggregation
Better progress logging

Add the Fastprop feature learner
Overhaul the way RelMT and Relboost generate features, making them more efficient

Significant improvement of project management:
- project.restart(), project.suspend(), and project.switch()
- multiple project support
Add custom __getattr__ and __dir__ methods to DataFrame, enabling column retrieval through autocomplete

Introduce new feature learner:
- RelMTModel [now RelMT],
- RelMTTimeSeries [now integrated in TimeSeries]

Add pipeline functionality: Pipeline, list_pipelines(), Features, Metrics, SQLCode, Scores
Better control of hyperparameter optimization: burn_in, kernels, optimization
Handling of time stamps: time
Improve database I/O: connect_odbc(), copy_table(), list_connections(), read_s3(), sniff_s3()
Enable S3 access: set_s3_access_key_id(), set_s3_secret_access_key()
New Feature Learner: MultirelTimeSeries, RelboostTimeSeries [now both integrated in TimeSeries]

Add XGBoostClassifier and XGBoostRegressor for improved predictive power
Overhaul of documentation
- Introduction of "getML in one minute" (now Quickstart) and "How to use this guide" (now User Guide)
- Introduction of User Guide (now Concepts) to include data annotation, feature engineering, hyperparameter optimization and more
Integration with additional databases like Greenplum, MariaDB, MySQL, and extended PostgreSQL support

Rename Autosql to Multirel
Boolean and categorical columns: Add support for boolean columns and operators, along with enhanced categorical column handling.
Introduce API improvements: fitting, saving/loading of models, data transformation
Add support for various aggregation functions such as MEDIAN, VAR, STDDEV, and COUNT_DISTINCT
Move from closed beta to pip
Introduce basic hyperopt algorithms: LatinHypercubeSearch, RandomSearch