Changes between Initial Version and Version 1 of Ticket #904
 Timestamp:
 20110228 12:58:43 (12 years ago)
Legend:
 Unmodified
 Added
 Removed
 Modified

Ticket #904 – Description
initial v1 1 We need to give users information about how to avoid over fitting when doing the time series fitting. For this, we need a new routine for the time series fitting form. There are two options for how to do this, a simple and a more complicated which involves a series of runs. I'll describe both (having worked on this all day, and having just arrived two hours late to Grand Rapids MI only to find out that the restaurant and bar closed two hours ago, and that this means no lunch no dinner today. Great).1 We need to give users information about how to avoid over fitting when doing the time series fitting. For this, we need a new routine for the time series fitting form. There are two options for how to do this, a simple and a more complicated which involves a series of runs. I'll describe both: 2 2 3 The simple version: 4 We calculate the AIC = 2 x number of search parameters + number of data points x ln (SS) 5 The number of search parameters is the number of vulnerabilities we estimate + the number of spline points + the number of years we find primary production anomalies for. 6 The number of data points is more tricky. In principle it is: 3 == The simple version == 4 We calculate the 7 5 8 Relative biomass = # data points –1 9 Absolute biomass = # data points 6 {{{ 7 AIC = 2 x number of search parameters + number of data points x ln (SS) 8 }}} 9 The number of search parameters is the number of vulnerabilities we estimate + the number of spline points + the number of years we find primary production anomalies for. The number of data points is more tricky. In principle it is: 10 11 {{{ 12 Relative biomass = # data points –1 13 Absolute biomass = # data points 10 14 Catches = # data points 15 }}} 16 This is however likely to overestimate the real number of datapoints because of correlation between them. Numbers in a data series are not independent observations. Carl suggests that we divide the number of datapoints with 5. I'm inclined to say that each time series gives us 12 data points. In any case, I'll give directions for what we do, still discussing. So, we need to make this an entry with a default value, which users can overrule. On the form, we thus need: Data points: ____ AIC = ____ The easy way is to just write the AIC in the iterations field after a search is done. 11 17 12 This is however likely to overestimate the real number of datapoints because of correlation between them. Numbers in a data series are not independent observations. Carl suggests that we divide the number of datapoints with 5. I'm inclined to say that each time series gives us 12 data points. In any case, I'll give directions for what we do, still discussing. So, we need to make this an entry with a default value, which users can overrule. On the form, we thus need: Data points: ________ AIC = ________ 13 The easy way is to just write the AIC in the iterations field after a search is done. 18 == The more complicated version == 19 see the attached spreadsheet. I made the analysis here, and it can be done fairly simple as a batch run. For this we need to run through: 14 20 15 The more complicated version: see the attached spreadsheet. I made the analysis here during my travel today, but it can be done fairly simple as a batch run. For this we need to run through: 21 First make a run with time series loaded, calculate SS, then AIC with no search parameters 16 22 17 First make a run with time series loaded, calculate SS, then AIC with no search parameters 23 If vulnerability is checked then Reset vulnerabilities If anomaly is checked then reset the forcing function in use 18 24 19 If vulnerability is checked then Reset vulnerabilities 20 If anomaly is checked then reset the forcing function in use 21 22 If vulnerability then first run sensitivity, and find out the order in which to include predators in search. 23 Search for vulnerability for the most sensitive predator; record SS and AIC; reset vulnerability ; search for the two most sensitive predators, etc until we search for all consumers. 25 If vulnerability then first run sensitivity, and find out the order in which to include predators in search. Search for vulnerability for the most sensitive predator; record SS and AIC; reset vulnerability ; search for the two most sensitive predators, etc until we search for all consumers. 24 26 25 27 If anomaly search is checked (but not vulnerability search): search for 0 spline points (done already, it's the same results as before we set any search parameters), then try 2, 3, 4, 5, 10, 15, 20, 25, 30, 35, up to number of years – 5. Then make an anomaly search with spline = 0, I.e. Search for primary production anomaly for all years. … … 28 30 29 31 Perhaps we can just dump the results from this to a csv file? 30