Expand description
Run flow as batching mode which is time-window-aware normal query triggered when new data arrives
Modules§
- engine 🔒
- Batching mode engine
- frontend_
client 🔒 - Frontend client to run flow as batching task which is time-window-aware normal query triggered every tick set by user
- state 🔒
- Batching mode task state, which changes frequently
- task 🔒
- time_
window 🔒 - Time window expr and helper functions
- utils 🔒
- some utils for helping with batching mode
Constants§
- DEFAULT_
BATCHING_ ENGINE_ QUERY_ TIMEOUT - TODO(discord9): make those constants configurable The default batching engine query timeout is 10 minutes
- FRONTEND_
ACTIVITY_ TIMEOUT - Frontend activity timeout if frontend is down(not sending heartbeat) for more than FRONTEND_ACTIVITY_TIMEOUT, it will be removed from the list that flownode use to connect
- FRONTEND_
SCAN_ TIMEOUT - Flow wait for available frontend timeout, if failed to find available frontend after FRONTEND_SCAN_TIMEOUT elapsed, return error which should prevent flownode from starting
- GRPC_
CONN_ 🔒TIMEOUT - Grpc connection timeout
- GRPC_
MAX_ 🔒RETRIES - Grpc max retry number
- MIN_
REFRESH_ DURATION - The minimum duration between two queries execution by batching mode task
- SLOW_
QUERY_ THRESHOLD - will output a warn log for any query that runs for more that 1 minutes, and also every 1 minutes when that query is still running