Skip to main content

Smart Assertions Stuck in Learning Mode Due to Incorrect Maximum Learning Period Configuration

Angelo Bresci
Updated

Area: Observability Issues

Sub-Area: Smart Assertions Training Configuration

Issue

Smart Assertions remain in learning mode indefinitely despite configuring a Maximum Learning Period (Maximum Training Data Days), with assertions showing training status even after exceeding the configured timeframe. This occurs when users set the Maximum Learning parameter to a value that is too low to meet the minimum training requirements, preventing assertions from graduating to active monitoring mode.

Error Messages

  • Something unexpected happened during training
  • Custom SQL query resulted in an error (while assertion continues to execute)
  • Connection timed out (during platform upgrades)

You Might Be Asking

  • Why does my assertion stay in learning mode when I set Maximum Learning to 5 days?
  • What does the Maximum Training Data Days parameter actually control?
  • How long should I expect Smart Assertions to remain in learning mode?
  • Why do I see error messages even though my assertion is working?

Solution

  1. Understand the parameter's actual function: The "Maximum Training Data Days" (Max Learning) parameter controls how far back the system looks for historical data to train the assertion model—it is a training data lookback window, not a deadline for exiting learning mode.
  2. Meet minimum training requirements: Assertions exit learning mode only when both conditions are met:
    • At least 10 data points have been collected
    • Data points span a minimum of 7 days
  3. Configure appropriate lookback window: Set Maximum Training Data Days to at least 14-30 days (recommended: 30 days, default: 365 days): 
    Maximum Training Data Days: 30 days (minimum recommended)
This ensures: 
- Wide enough historical window to collect 10+ data points
- Data spans at least 7 days (required minimum)
- Both Volume and Freshness assertions can successfully train
  4. Reset existing assertions with low values: If you've already set the value too low (e.g., 5 days): 
    1. Navigate to the assertion configuration
2. Change Maximum Training Data Days back to 30+ days
3. Save the configuration
4. Wait for the assertion to complete training with sufficient data
  5. For persistent error displays: If assertions show errors but continue working: 
    1. Restart the affected assertions from the DataHub UI
2. Upgrade Remote Executor to version 0.3.17.2 or later
3. Monitor for automatic error state reset in newer versions
  6. Special considerations for Freshness assertions: These require a minimum of 10 recorded table operations over a 7-day span. If tables don't update frequently, training may take longer even with proper configuration.

Additional Notes

Setting Maximum Learning to less than 7 days creates a configuration conflict where the system cannot satisfy the minimum 7-day span requirement. This is a known bug where changing Maximum Training Data Days before model completion and setting it too low prevents training from finishing. The issue has been identified and will be fixed in future releases. Error messages may persist in the UI even after successful execution due to state persistence issues, particularly after platform upgrades or connection timeouts. The assertion functionality remains unaffected.

Related Documentation

Tags: smart-assertions, learning-mode, training-configuration, maximum-learning-days, observability, assertion-training, volume-assertions, freshness-assertions, error-display, remote-executor