# Adaptive alpha\_regret: Objective standard for choosing alpha\_regret to best handle changing worker performance **URL:** \ **Category:** Inference Synthesis\ **Created:** [August 9, 2024, 2:42pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74 "2024-08-09T14:42:05Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 9, 2024, 2:42pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/1 "2024-08-09T14:42:05Z")
A follow-up on [the post “Determine guidelines for choosing alpha\_regret”](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32) Imagine worker gets much better or much worse. We don’t want to overreact if that is incidental, but we want to catch it early if it is systematic. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 9, 2024, 3:02pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/2 "2024-08-09T15:02:40Z")
To summarize, our goal is to set the parameter α\_regret for each worker individually, letting α\_regret evolve over time based on the worker’s reliability. Initially, we might start with α\_regret being set to 0.1, adjusting it as needed to potentially reach equilibrium. **Background & intuition:** α\_regret is a parameter in the Exponential Moving Average (EMA) that is inversely proportional to a time constant τ, which is a period it takes for the EMA to reflect approximately 63.2% of a step change in the input data (mathematically, it’s a random variable). We will refer to τ as the window size. It is logical for τ to depend on systematic fluctuations in a worker’s regret. For instance, in scenarios where there is gradual improvement from a cold start, maintaining a small α\_regret would result in the network retaining the cold start effects for an extended duration. Therefore, we’d expect α\_regret to increase with improvements in worker performance. To adjust α\_regret, let’s consider a measure of volatility of the current regrets. If it is high, we should opt for a smaller α\_regret to avoid overemphasizing temporary outperformance. Conversely, if a worker has consistently shown improvement after an initial rough start, we should increase α\_regret to give more weight to their recent performance. **Computational Specifics:** Given the impracticality of storing regrets data from all previous epochs, we adopt an Exponential Weighted Moving Average (EWMA) approach. We define the volatility for each epoch i as follows: volatility\_i = [λ \* r\_i^p\_regret + (1 - λ) \* volatility\_(i-1))^p\_regret]^(1/p\_regret), where r\_i = ([current\_regret at epoch i-1] - [current\_regret at epoch i])\_+ = max ([current\_regret at epoch i-1] - [current\_regret at epoch i], 0)\*. This formulation exclusively captures the reduction in current regret over a single epoch, as an increase in regret is advantageous and should not be subject to penalization. Indeed, greater emphasis should be placed on workers whose predictions consistently improve over time, rather than on those whose performance remains static. \*Also note that we refer to the log loss difference in equation (15) in the Whitepaper as “current\_regret”. That is so that we (a) capture the most recent change in worker performance, unaffected by historical data, and (b) avoid influencing the calculated value of volatility\_i, since it will influence the value of α\_regret, that is used in the calculation in equation (15) to determine \cal R\_il. ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/9b03a08927b43b089078d5a4f9bc64eec2fe9f49.png) To proceed, we need to establish optimal values for the parameters λ and p\_regret. I propose initiating our experiments with p\_regret set to 2, as it represents a natural starting point. The choice of λ depends on the data frequency and desired responsiveness. After selecting a reasonable value for λ, our next step will be to define a functional form for the dependency of α\_regret on volatility. Right now our only insight is that it should be a decreasing function with values ranging between 0 and 1. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 10, 2024, 1:39am UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/3 "2024-08-10T01:39:26Z")
Focusing on λ for now, our initial simulator setup tracks the volatility and r\_i over the last 100 epochs, labeled as “Deviation” and “Returns” respectively on the plots. The plots below display these metrics for each worker across various λ values, using a log-scale to enhance readability. However, we intend to revise the simulator design to more accurately model sudden changes in workers’ behavior. Based on these observations, a λ value between 0.1 and 0.3 seems to be a reasonable choice. ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/bafbf914480f84fc6a0f5a20f02840d20e234cff.jpeg) ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 12, 2024, 12:51pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/4 "2024-08-12T12:51:40Z")
After testing various potential functional forms for α\_regret, we’ve identified an issue: α\_regret can only be sensitive to the changes of volatility within a certain range. But given the uncertainty in the expected volatility ranges, we’re now trying out a new approach to handle systematic changes in network participant behavior. Instead of setting a fixed range, we are adapting the function α\_regret(volatility) itself. Controlling the ranges is important because if α\_regret values become too extreme, they can skew how the network evaluates different workers, impacting the network performance. To address this, we are developing a dynamic form of α\_regret that adjusts according to the typical volatility range of the workers. Specifically, one needs to define a transformation that takes a volatility value and outputs an appropriate α\_regret value. Initially, we start by using a simple piece-wise linear (see the fig. below), which we will refer to as transformation ‘g’. ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/5924e499637d52da20d49199907fc0d0bf7e9e71.png) A few remarks in order: We’re looking at regrets = log loss differences, hence, we’d expect similar changes in regret (on the same scale) for the same type of worker, whether they are inferers or forecasters. Initially, I thought about assigning each worker a typical range, but then this could make α\_regret too sensitive to individual volatility spikes, which is unnecessary when, say, everyone is having a bad day. This led me to consider using EMA for α\_regret as well, to smooth out these fluctuations and avoid overreactions. Another argument in favor of using EMA for α\_regret: for small networks range based solely on current volatility can lead to extreme and noisy α\_regret values. So maybe we should include volatilities from the previous epoch: that would broaden the data set and smoothen the ranges. However, if the current volatilities are higher/lower than the previous volatilities, this would result in new α\_regret values being lower/higher, respectively, than if the range was determined based solely on the current volatilities. So again, EMA approach would be useful to smooth out /mitigate this. We’ve also updated the simulator setup to better model the fluctuations we’re trying to address. Specifically, we added “good/bad days” for each worker to simulate more realistic behavior. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 14, 2024, 3:35pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/6 "2024-08-14T15:35:48Z")
- `Dynamic volatility` is updated through a p\_normalized EMA ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/e849094c7f34655bc775a1ad89635965d34deab3.png) `α_regret` is also updated using EMA, but in the log-space. ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/eb948e79fce5a9efe82716c33019f09192e82c66.png) - `g` is a function of `Dynamic volatility`, it is piecewise linear: it is linearly decreasing, from a specified upper limit on `α_regret` to a lower limit, within the `volatility range`. It is constant outside of this range. ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/5924e499637d52da20d49199907fc0d0bf7e9e71.png) - The `volatility range` is defined by min/max volatility values at epoch for each type of worker. I.e., it is calculated separately for inferers and forecasters. We have adopted upper and lower bounds on alpha\_regret based on the experiments for non-dynamic alpha\_regret. We then vary parameters that define how volatility is computed, and compare the network’s prediction losses by averaging across different simulator runs. Additionally, we include a datapoint for a default setup with non-dynamic alpha\_regret to serve as a baseline for comparison. ![image (1)](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/3cc9e3c5b46d697e9ec77d05b7385a5fec25f744.png) **Conclusion:** There is no statistically significant benefit in using adaptive alpha regret. I suspect this stems form the fact that there is no difference in the network performance when varying alpha\_regret within the range 0.1, 0.01 and 0.001 in the experiments above. Hence, varying dynamic alpha\_regret within that range is not going to do much either. ![image (2)](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/67b625d7e59ad2eaea175a3aa2916d89cfc6636d.png) We conclude that introducing dynamic alpha\_regret in the current setting is redundant. Specifically, the performance is not affected when non-dynamic alpha\_regret is varied on a large scale, hence, dynamically varying alpha\_regret in the same range seems futile. At this point we don’t know what sort of behavior to expect from the network participants in the live system. Therefore, our recommendation is to monitor the live network for a while, and, if needed, we can explore further any possible benefit of using adaptive alpha\_regret. A simple diagnostic would be to test the network performance using multiple values of alpha\_regret. Significant differences might indicate the need for an adaptive alpha\_regret that is sensitive to historical volatility. ---
**Author:** ![Apollo11](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/apollo11/32/354_2.png) [@Apollo11](https://research.allora.network/u/Apollo11)\ **Post date:** [August 14, 2024, 5:50pm UTC](https://research.allora.network/t/adaptive-alpha-regret-objective-standard-for-choosing-alpha-regret-to-best-handle-changing-worker-performance/74/7 "2024-08-14T17:50:03Z")
Thank you for the careful analysis. I think your conclusion makes a lot of sense. Further network monitoring will help us assess under which circumstances an adaptive `alpha_regret` might boost network performance.