Commit Graph

59 Commits

Author SHA1 Message Date
chrislu
02d36637de limit with offset 2025-09-04 09:13:54 -07:00
chrislu
20f61388bf support limit with offset 2025-09-04 08:44:29 -07:00
chrislu
69e6902072 handling errors 2025-09-03 17:42:30 -07:00
chrislu
323193cf8c no more mysql 2025-09-03 15:34:39 -07:00
chrislu
50040a68bb fix 2025-09-03 09:54:31 -07:00
chrislu
c10a0ba2fd fmt 2025-09-03 08:11:32 -07:00
chrislu
72d332a352 feat: Add window function foundation with timestamp support
Added comprehensive foundation for SQL window functions with timestamp analytics:

Core Window Function Types:
- WindowSpec with PartitionBy and OrderBy support
- WindowFunction struct for ROW_NUMBER, RANK, LAG, LEAD
- OrderByClause for timestamp-based ordering
- Extended SelectStatement to support WindowFunctions field

Timestamp Analytics Functions:
 ApplyRowNumber() - ROW_NUMBER() OVER (ORDER BY timestamp)
 ExtractYear() - Extract year from TIMESTAMP logical type
 ExtractMonth() - Extract month from TIMESTAMP logical type
 ExtractDay() - Extract day from TIMESTAMP logical type
 FilterByYear() - Filter records by timestamp year

Foundation for Advanced Window Functions:
- LAG/LEAD for time-series access to previous/next values
- RANK/DENSE_RANK for temporal ranking
- FIRST_VALUE/LAST_VALUE for window boundaries
- PARTITION BY support for grouped analytics

This enables sophisticated time-series analytics like:
- SELECT *, ROW_NUMBER() OVER (ORDER BY timestamp) FROM user_events WHERE EXTRACT(YEAR FROM timestamp) = 2024
- Trend analysis over time windows
- Session analytics with LAG/LEAD functions
- Time-based ranking and percentiles

Ready for production time-series analytics with proper timestamp logical type support! 🚀
2025-09-03 07:33:31 -07:00
chrislu
699e2f4413 feat: Add logical type support to SQL query engine
Extended SQL engine to handle new Parquet logical types:
- Added TimestampValue comparison support (microsecond precision)
- Added DateValue comparison support (days since epoch)
- Added DecimalValue comparison support with string conversion
- Added TimeValue comparison support (microseconds since midnight)
- Enhanced valuesEqual(), valueLessThan(), valueGreaterThan() functions
- Added decimalToString() helper for precise decimal-to-string conversion
- Imported math/big for arbitrary precision decimal handling

The SQL engine can now:
-  Compare TIMESTAMP values for filtering (e.g., WHERE timestamp > 1672531200000000000)
-  Compare DATE values for date-based queries (e.g., WHERE birth_date >= 12345)
-  Compare DECIMAL values for precise financial calculations
-  Compare TIME values for time-of-day filtering

Next: Add YEAR(), MONTH(), DAY() extraction functions for date analytics.
2025-09-03 07:29:03 -07:00
chrislu
ec1e74a6e8 feat: Add Parquet logical types to mq_schema.proto
Added support for Parquet logical types in SeaweedFS message queue schema:
- TIMESTAMP: UTC timestamp in microseconds since epoch with timezone flag
- DATE: Date as days since Unix epoch (1970-01-01)
- DECIMAL: Arbitrary precision decimal with configurable precision/scale
- TIME: Time of day in microseconds since midnight

These types enable advanced analytics features:
- Time-based filtering and window functions
- Date arithmetic and year/month/day extraction
- High-precision numeric calculations
- Proper time zone handling for global deployments

Regenerated protobuf Go code with new scalar types and value messages.
2025-09-03 07:18:58 -07:00
chrislu
d60c542ecc feat: Replace pg_query_go with lightweight SQL parser (no CGO required)
- Remove github.com/pganalyze/pg_query_go/v6 dependency to avoid CGO requirement
- Implement lightweight SQL parser for basic SELECT, SHOW, and DDL statements
- Fix operator precedence in WHERE clause parsing (handle AND/OR before comparisons)
- Support INTEGER, FLOAT, and STRING literals in WHERE conditions
- All SQL engine tests passing with new parser
- PostgreSQL integration tests can now build without CGO

The lightweight parser handles the essential SQL features needed for the
SeaweedFS query engine while maintaining compatibility and avoiding CGO
dependencies that caused Docker build issues.
2025-09-03 07:11:18 -07:00
chrislu
4d9de40c5c fmt 2025-09-03 00:48:09 -07:00
chrislu
42661ac110 fix tests 2025-09-03 00:47:08 -07:00
chrislu
991247facf fix tests 2025-09-03 00:40:03 -07:00
chrislu
e3e369c264 change to pg_query_go 2025-09-03 00:10:47 -07:00
chrislu
ba4a8b91d5 fmt 2025-09-02 22:31:53 -07:00
chrislu
59d6806146 fix empty spaces and coercion 2025-09-02 22:30:52 -07:00
chrislu
3fa7670557 fix todo 2025-09-02 22:12:47 -07:00
chrislu
687c5d6bfd fix tests 2025-09-02 21:21:59 -07:00
chrislu
a7eb178cec Update engine.go 2025-09-02 18:37:31 -07:00
chrislu
60066a6a4c read broker, logs, and parquet files 2025-09-02 18:15:26 -07:00
chrislu
d29f54e0be de-support alter table and drop table 2025-09-02 17:23:46 -07:00
chrislu
8e15fdf2c7 remove sample data 2025-09-02 17:18:19 -07:00
chrislu
f776a49322 avoid sample data 2025-09-02 17:15:13 -07:00
chrislu
ed7102df6e column name can be on left or right in where conditions 2025-09-02 17:05:31 -07:00
chrislu
586a795b02 return fast on error 2025-09-02 14:09:38 -07:00
chrislu
c9e093194d setup integration test for postgres 2025-09-02 10:51:37 -07:00
chrislu
6fb88a8edb buffer start stored as 8 bytes 2025-09-02 00:57:48 -07:00
chrislu
db75742e37 explain with broker buffer 2025-09-02 00:35:52 -07:00
chrislu
467034c8c7 process buffer from brokers 2025-09-01 23:45:52 -07:00
chrislu
7ca3b59c44 save source buffer index start for log files 2025-09-01 23:33:35 -07:00
chrislu
f5ed25f755 fix decoding data 2025-09-01 22:17:17 -07:00
chrislu
99bfe95e51 detailed logs during explain. Fix bugs on reading live logs. 2025-09-01 22:16:46 -07:00
chrislu
c7a0b89067 fix after refactoring 2025-09-01 21:40:24 -07:00
chrislu
e385f0ce7d refactor 2025-09-01 20:19:59 -07:00
chrislu
61ad3c39ac add tests 2025-09-01 20:05:24 -07:00
chrislu
4214d765cf use mock for testing 2025-09-01 19:36:47 -07:00
chrislu
a3f6a5da27 skip 2025-09-01 18:55:14 -07:00
chrislu
dfd0897e49 improve tests 2025-09-01 18:52:22 -07:00
chrislu
7d88a81482 add tests 2025-09-01 18:00:55 -07:00
chrislu
eaa7136c92 explain the execution plan 2025-09-01 17:19:36 -07:00
chrislu
93a09f5da4 explain 2025-09-01 16:47:52 -07:00
chrislu
55cad6dc4a combine parquet results with live logs 2025-09-01 16:26:35 -07:00
chrislu
e3798c2ec9 sql 2025-09-01 16:12:10 -07:00
chrislu
55dfb97fc8 parquet file generation remember the sources also 2025-09-01 15:11:06 -07:00
chrislu
2fa8991a52 scan all files 2025-09-01 14:51:23 -07:00
chrislu
c7598d89f1 remove emoji 2025-09-01 13:55:44 -07:00
chrislu
c73ceac79f use parquet statistics for optimization 2025-09-01 13:50:56 -07:00
chrislu
471ba271dc fix reading system fields 2025-09-01 13:28:38 -07:00
chrislu
8645f3a264 column name case insensitive, better auto column names 2025-09-01 11:25:04 -07:00
chrislu
32e73811f2 support aggregation functions 2025-09-01 11:15:00 -07:00