# Determine guidelines for choosing alpha\_regret **URL:** \ **Category:** Inference Synthesis\ **Created:** [June 17, 2024, 9:38am UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32 "2024-06-17T09:38:05Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Apollo11](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/apollo11/32/354_2.png) [@Apollo11](https://research.allora.network/u/Apollo11)\ **Post date:** [June 17, 2024, 9:38am UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/1 "2024-06-17T09:38:05Z")
`alpha_regret` is a free parameter chosen by topic creators. We need to understand how the network loss depends on `alpha_regret` so topic creators can choose a sensible, and ideally optimal, value. The first stage of the study is to take the default simulated network and finely vary `alpha_regret` to see how the loss changes. Are there any trends, minima, or is there just scatter? The output metrics should focus on the final ~100 epochs to ensure the results are not skewed by the burn-in (getting rid of the cold-start values of regrets, etc.). It is important to use _log_-averages or medians of the loss. Depending on how the study develops, possible extensions are to see how the results change with different compositions of network participants. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [June 17, 2024, 9:09pm UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/2 "2024-06-17T21:09:25Z")
**Preliminary analysis: varying alpha\_regret; other parameters at default** Recall that alpha\_regret is a parameter for estimating the workers’ regret, defined as: R\_il = alpha\_regret \* current\_regret + (1-alpha) \* historical\_regret We begin with a preliminary study examining the relationship between network loss and the alpha\_regret parameter while keeping other parameters at their default settings. Specifically, we train the network over 1000 epochs and compare the performance over the last 100 epochs. The alpha\_regret parameter is varied between 0 and 1 in increments of 0.1. As a first step, we plot the log-averages and log-medians of the data. We observe that alpha\_regret = 0 (when all the regret is historical, i.e., the regret does not change) is an obvious outlier. ![image_480](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/6126746fb5fa733446bb6b91ef243823e30d1918.png) ![image_720](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/a6d1b864e2347690bd30c00ef1765ed79a3f22ee.png) On the plot above, the shaded area around the medians represents the 95% CI’s, where the margin of error is given by the formula 1.96\*std/sqrt(100). Taking the results as they are, we observe a decrease in loss when we introduce current regret (a jump from alpha\_regret being 0 to 0.1). Beyond this point, the differences are not statistically significant. For more finely grained alpha\_regret sampling, training the network over 500 epochs: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/44d75c92153372b9439a3648f5a577546bd51072.png) The trend is flat for alpha\_regret \> 0, and there seems to be no dependence of std on alpha\_regret. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [June 19, 2024, 6:13pm UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/3 "2024-06-19T18:13:38Z")
Since alpha\_regret is a parameter in the exponential moving average, let’s vary alpha\_regret on a logarithmic scale. We obtain the following: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/68059fa91c71d6a53e0ccf10128eeccf9057542e.png) Zooming in on the range between 10^-2 and 10^-4, the dependence turns out to be piecewise linear: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/43d5413b4e590a9d5cb17c7a21743db22d43cbf7.png) It seems that the optimal choice of alpha\_regret lies somewhere between 10^(−2.8) and 10^(−2.9) at the default network parameters. When varying other network parameters, we observe that the transition typically occurs between 10^(−3.5) and 10^(−3.0), similar to before, except for an outlier network with n\_predictors=3: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/6a37eb682ceb2666eb0911d58bd43ab7096af1bf.png) ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/3bbeaceeabca4104553baffd6d548cc9f2a8df39.png) ---
**Author:** ![Apollo11](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/apollo11/32/354_2.png) [@Apollo11](https://research.allora.network/u/Apollo11)\ **Post date:** [June 19, 2024, 7:21pm UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/4 "2024-06-19T19:21:52Z")
Nice! Could the transition at alpha\_regret = 1e-3 be because we run for 1000 epochs? In other words, for alpha\<1e-3 == 1/n\_epochs, the network would not get rid of the cold start signal before the end of the run. I think that’s what we’re seeing. So basically it does not matter for overall network performance what alpha\_regret we use, but it does matter for how quickly we get rid of the initial conditions? ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [June 22, 2024, 12:48am UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/5 "2024-06-22T00:48:42Z")
Indeed, our observations demonstrate that the time constant (= the amount of time it takes for the exponential moving average to reflect approximately 63.2% of a step change in the input data) is proportional to `1/α_regret`​. Hence, when running the network over 1000 epochs with smaller `α_regret` values, the influence of a cold start remains significant till the end, and the weights assignment is suboptimal. However, once we reach equilibrium, the initial conditions’ impact diminishes, and overall network performance doesn’t depend on the exact value of `α_regret`. ---
**Author:** ![Apollo11](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/apollo11/32/354_2.png) [@Apollo11](https://research.allora.network/u/Apollo11)\ **Post date:** [June 22, 2024, 11:10am UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/6 "2024-06-22T11:10:28Z")
OK great – this shows that the role of `alpha_regret` is really to handle changing worker performance. Imagine a worker suddenly gets much better or much worse. The network shouldn’t overreact if that is incidental, but should nonetheless catch it early if it is systematic. So maybe we want to start thinking about some objective standard to dynamically change `alpha_regret` as a function of changing worker performance. This should improve network performance for non-static worker properties. ---
**Author:** ![steve](https://avatars.discourse-cdn.com/v4/letter/s/f19dbf/32.png) [@steve](https://research.allora.network/u/steve)\ **Post date:** [June 28, 2024, 5:44am UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/7 "2024-06-28T05:44:01Z")
Very interesting. I agree, dynamically varying `alpha_regret` seems like a good approach here. ---
**Author:** ![Renata](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/renata/32/61_2.png) [@Renata](https://research.allora.network/u/Renata)\ **Post date:** [August 14, 2024, 1:36pm UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/8 "2024-08-14T13:36:55Z")
We now removed the EMA from the reputers in the simulator, so I redid the test and this is what it looks like now — the default alpha=0.1 is great: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/7c8d51d3f80ef522ecfaaf9ec143f2da0bf9f82e.png) Additionally, we test that over multiple seeds to show statistical significance of our results. A box shows 10 median `log(combintor_loss)` values from a single network run: ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/a7ee1d14e276fee82408163211fb05c785153ca8.png) ![image](https://canada1.discourse-cdn.com/flex003/uploads/allora/original/1X/366cc8118a0f4592963dfc4ae47de88d18b77d7f.png) ---
**Author:** ![Apollo11](https://yyz1.discourse-cdn.com/flex003/user_avatar/research.allora.network/apollo11/32/354_2.png) [@Apollo11](https://research.allora.network/u/Apollo11)\ **Post date:** [August 14, 2024, 2:45pm UTC](https://research.allora.network/t/determine-guidelines-for-choosing-alpha-regret/32/9 "2024-08-14T14:45:41Z")
This is great! Thank you for sharing.