Unverified Commit 46bbdf64 authored by Simon Bowly's avatar Simon Bowly
Browse files

Cleanup

parent 92fbe4a3
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
%% Cell type:markdown id: tags:
# Regression Workflow (Diabetes)
%% Cell type:markdown id: tags:
In this lesson we will go over the workflow for regression modelling. This investigates the effect of various features on the tendency of patients to develop diabetes, using the `sklearn` methods `LinearRegression` and `Ridge` regularization. We will look at how coefficients can vary signficantly due to overfitting, and how that can be alleviated using regularization.
%% Cell type:markdown id: tags:
## Import Modules and Data
%% Cell type:markdown id: tags:
First import the Diabetes dataset. The same dataset can be imported from the `sklearn` example datasets, but is already normalized. We will use the unnormalized dataset initially.
%% Cell type:code id: tags:
``` python
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split # for splitting the data into training and testing sets
from sklearn.linear_model import LinearRegression, Ridge, Lasso # models we are going to use
from sklearn.metrics import r2_score # for comparing the predicted and test values
```
%% Cell type:code id: tags:
``` python
df = pd.read_csv('Diabetes_Data.csv') # read the Diabetes dataset in to a pandas dataframe
```
%% Cell type:markdown id: tags:
## Data Analysis
%% Cell type:markdown id: tags:
`BP` is blood pressure, `BMI` is body mass index (healthy range for adults is 18.5 to 25) and `S1`-`S6` are various blood serum measurements. `Y` is the target variable, and is a measure of disease progression one year after the original measurements. Aim is to predict `Y` from the other variables.
We can look at the first few values of the variables, and at the descriptive statistics.
%% Cell type:code id: tags:
``` python
df.head()
```
%%%% Output: execute_result
AGE SEX BMI BP S1 S2 S3 S4 S5 S6 Y
0 59 2 32.1 101.0 157 93.2 38.0 4.0 4.8598 87 151
1 48 1 21.6 87.0 183 103.2 70.0 3.0 3.8918 69 75
2 72 2 30.5 93.0 156 93.6 41.0 4.0 4.6728 85 141
3 24 1 25.3 84.0 198 131.4 40.0 5.0 4.8903 89 206
4 50 1 23.0 101.0 192 125.4 52.0 4.0 4.2905 80 135
%% Cell type:code id: tags:
``` python
df.describe()
```
%%%% Output: execute_result
AGE SEX BMI BP S1 S2 \
count 442.000000 442.000000 442.000000 442.000000 442.000000 442.000000
mean 48.518100 1.468326 26.375792 94.647014 189.140271 115.439140
std 13.109028 0.499561 4.418122 13.831283 34.608052 30.413081
min 19.000000 1.000000 18.000000 62.000000 97.000000 41.600000
25% 38.250000 1.000000 23.200000 84.000000 164.250000 96.050000
50% 50.000000 1.000000 25.700000 93.000000 186.000000 113.000000
75% 59.000000 2.000000 29.275000 105.000000 209.750000 134.500000
max 79.000000 2.000000 42.200000 133.000000 301.000000 242.400000
S3 S4 S5 S6 Y
count 442.000000 442.000000 442.000000 442.000000 442.000000
mean 49.788462 4.070249 4.641411 91.260181 152.133484
std 12.934202 1.290450 0.522391 11.496335 77.093005
min 22.000000 2.000000 3.258100 58.000000 25.000000
25% 40.250000 3.000000 4.276700 83.250000 87.000000
50% 48.000000 4.000000 4.620050 91.000000 140.500000
75% 57.750000 5.000000 4.997200 98.000000 211.500000
max 99.000000 9.090000 6.107000 124.000000 346.000000
%% Cell type:markdown id: tags:
Since the maximum and minumum values of all features are similar, that suggests there are no missing data points and we don't need to clean the data.
Here the field `SEX` only has two values.
%% Cell type:code id: tags:
``` python
df['SEX'].unique()
```
%%%% Output: execute_result
array([2, 1])
%% Cell type:markdown id: tags:
Categorical data such as this should be converted to binary columuns using the `sklearn` method `OneHotEncoder`. However, since `SEX` is already binary, there is no need to do this. If we had a field which had three categories, for example, the data came from three different states, then we could create a binary field for each state, with the value 1 if the person is resident in the state and 0 otherwise. For more details see `sklearn.feature_extraction.DictVectorizer` and `sklearn.preprocessing.OneHotEncoder`.
We can look at the variable correlations to search for patterns. `SEX` doesn't seem to be important in predicting `Y`, and `AGE`, `S1` and `S2` are only marginally important. The most important variables seem to be `BMI`, `BP`, `S4` and `S5`. There is a strong correlation between `S1` and `S2`, and `S3` and `S4`.
%% Cell type:code id: tags:
``` python
corrs = df.corr() # calculate the correlation table
# as this is a symmetric table, set up a mask so that we only plot values below the main diagonal
mask = np.triu(np.ones_like(corrs, dtype=np.bool))
f, ax = plt.subplots(figsize=(10, 8)) # initialise the plots and axes
# plot the correlations as a seaborn heatmap, with a colourbar
sns.heatmap(corrs, mask=mask, center=0, annot=True, square=True, linewidths=.5)
# do some fiddling so that the top and bottom are not obscured
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5);
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiMAAAHSCAYAAAA+DMuQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1R7G8e/ZBKQjNaFJEUE6XqrSBZSuAoqoCDawgIgFpYmIioKiIoKCImADpChVelWqIr2FHtLovWVz7h8JIT2B7GZX9/3cZ59nZ+Zs5t0h3v3llFljrUVERETEUxyeDiAiIiK+TcWIiIiIeJSKEREREfEoFSMiIiLiUSpGRERExKNUjIiIiIhH+WfAObR2WERE/u1Mhp5t2zT3fHZWaJex7yON1DMiIiIiHqViRERERDxKxYiIiIh4VEbMGREREZEbYJ1Ot/xcr5wwgnpGRERExMPUMyIiIuJtnJGeTpChVIyIiIh4GRvlnmJEwzQiIiIiSVDPiIiIiLdx0wRWb6WeEREREfEo9YyIiIh4GasJrCIiIuJRPlaMaJhGREREPErFiIiIiHiUihERERHxKM0ZERER8TLuuumZt1IxIiIi4m10n5Foxpg74zy/JcGx2u4MJSIiIr4jpZ6Rn4D/xTxfHec5wKgE2yIiIuIivnafkZQmsJpknie1Hf+gMV2NMRuMMRvGjBlz0+FERETkvy+lnhGbzPOktuMftHYMMCYtbUVERCQBD/WMGGOaAZ8DfsA31toPExwvDowDCgAngCestcHpPW9KxUhRY8wIontBrj0nZrtIek8sIiIi3sMY4wd8CTQFgoH1xpiZ1trtcZp9DEy01k4wxtwLDAE6pffcKRUjb8R5viHBsYTbIiIi4iI2yiOraWoCQdbafQDGmEnAA0DcYqQ80Cvm+VLgV1ecONlixFo7wRUnEBERkRvjoQmsRYDDcbaDgVoJ2mwC2hE9lPMQkNMYk89aezw9J05paW9dY8yTcbanGmOWxDzuTc9JRUREJOPFXWAS8+ga93ASL0k47/N1oIExZiPQADgCpLtySmmYZhDQI852WaALkB3oCyxJ78lFREQkCW7qGUmwwCShYKBYnO2iQEiC14cAbQGMMTmAdtba0+nNldLS3lwJJq3ssdb+Za1dAeRM74lFRETEq6wH7jDGlDTGZAYeBWbGbWCMyW+MuVY79CF6ZU26pVSM3Bp3w1rbNs5mgCtOLiIiIt7BWhsJdAfmAzuAKdbabcaYd40xbWKaNQR2GWN2E10LvO+Kc6c0TLPTGNPSWjsn7k5jTCtglytOLiIiIol5aDUN1tq5wNwE+96O83wqMNXV502pGOkFzDHGtAf+jtlXDbgHaOXqICIiIuKbUlraG2SMqQw8DlQgekbtcuAboCfwUoYkFBER8TU+9t00KfWMYK29DIwzxtwFdAQGAvuBaRmQTURERHxAssWIMaYM0TNpOwLHgcmAsdY2yqBsIiIiPsnXvrU3xQmswEqgtbU2CMAY0yuF9iIiIuIKPlaMpLS0tx0QBiw1xow1xjQm6buziYiIiNy0lCawzgBmGGOyAw8SvbomwBgzGphhrV2QQRlFRER8iqeW9npKSj0jAFhrz1trf7TWtiL61rD/AG+5PZmIiIj4hBRX0yRkrT0BfB3zEBEREXfwsTkjN1SMiIiIiPtZp28N02RIMdKkdIWMOM0NWxS0zdMRREREfF6qc0ZERERE3EnFiIiIiHiU5oyIiIh4GV+7A6t6RkRERMSj1DMiIiLibaJ8q2dExYiIiIiX8bWlvRqmEREREY9Sz4iIiIi3Uc+IiIiISMZRz4iIiIiX8bWlvSpGREREvI2GaUREREQyjnpGREREvIyW9oqIiIhkIK8pRmrUr8t3C2YzYfE8Hu32bKLjlWpUY/RvvzB/5ybqNbsvdn+V2jX5aua02MfcbX9zT5N7MzK6iIiIpINXDNM4HA56vNOPNzs/x9GwcL6cPpk/Fy/lUNDe2DYRIaEM7d2PR57tEu+1m9as4/k27QDImTs3ExbP469Vf2ZkfBEREZeyUb41TOMVxUjZKpUIOXiY0MPBACybM5c6TRrFK0bCj4QAEBVlk/059Zvdx/rlK7l86ZJ7A4uIiIjLpDhMY4z5whiTM4n9dxpjFrkqRP6AACJCQ2O3j4aFky8g4IZ/TsNWzVkye66rYomIiHiG0+meh5dKbc5IGPCPMeYxAGNMNmPMUGAm8GVyLzLGdDXGbDDGbBgzZkyqIYxJYqdNvgckKXkL5Kdk2TvYsPKPG3qdiIiIeFaKwzTW2veNMT8BI40xzwOFgSlAVWvthRReNwa4VoXYKUM/TzHE0bBwChYqFLtdIDCA4xERaXsHMRq0aMYfCxbjjPStu9aJiMh/j5b2Jnati8I/pv2OlAqRm7Fr81aKFL+NwKJF8M+UiYYtW/Dn4qU39DPubd1CQzQiIiL/QqnNGekPLAImWmvvAeoBDxhjlhtjyrsqRJTTyReD3ufD78Ywbv5Mls/9nYN79tK5Z3fubtwIgLKVKvLzqsXUb34fvQYP5Jt5v8W+PqBIYQoEBrJ57XpXRRIREfEY64xyy8NbGZvC3AxjzOdAf2vt2QT7mwPDrbXl0nAO26R0hfSldJNFQds8HUFERP4dkprd6DYn+je4sYmTaZT3veUZ+j7SKsWeEWttz4SFSMz+eUBVt6USERERn5HaMM2UOM8/SnB4llsSiYiI+DjrdLrl4a1Sm8B6R5znTRMcK+DiLCIiIuKDUrsDa0pjVm4ZzxIREfF11ulbH7GpFSPZjDF3Ed2DktUY87+Y/QbI6tZkIiIi4hNSK0ZCgU+ILj7CgI/jHAtzVygRERHxHakVI28Ch621oQDGmM5AO+AA8I5bk4mIiPgob74niDukNoH1K+AygDGmPjAEmACc5vrt3kVERERuWmo9I37W2hMxzzsAY6y104Bpxph/3BtNRETEN/laz0iqxYgxxt9aGwk0BrrewGtFRETkJtgoraaJ62dguTHmGHARWAlgjClN9FCNiIiISLqkWIxYa983xiwGCgEL7PUvsnEAPdwdTkRExBfpPiMJWGvXJLFvt3viiIiIiK/RvA8REREvY733a2TcIkOKkUVB2zLiNCIiIvIvpJ4RERERL6M5I27Q/I5KGXGaGzZvzxYqlyjl6RjJ2nxgn6cjiIiIB0T51m1GUr0Dq4iIiIhbqRgRERERj1IxIiIiIh6lCawiIiJeRkt7RURExKN8rRjRMI2IiIh4lHpGREREvIyW9oqIiIhkIPWMiIiIeBlfmzOiYkRERMTLREUZT0fIUBqmEREREY9Sz4iIiIiX0QRWERERkQzkNcVItXp1GDt/Jt8umsPDXZ9JdLxijWp88etkZu/YSN1mTeMde/qNXoyeM53Rc6ZTv8X9LstUp0F9Zi5exOxlS3j6hecTHc+UOTNDR45g9rIl/PjrdAoXLQKAv78/730yjGm/z+PXRQt45sUXYl/zxDNPM33B70yfP4+PRnxO5lsyuyyviIjIv5FXFCMOh4OX3unHgGdfpFvzB2jYqjm3lS4Vr01ESCifvDmApbPmxttfo2E9bq9QjpfaPMwr7R+n3bNPkS1Hdpdk6vvuIF7o8hQPNr2f5m1aU6p06Xht2j7yCGdOn6FVw3v5/ttxvPLWmwDc16IFmTJnpl2z5jzaqg3tH+tI4aJFKBgQwONdOtOx9QO0vb85DoeDZq1bpzuriIjIv5lXFCNlKlci5OAhwg4HE3k1kuVz5lG7caN4bSKOhHBg126stfH231b6dras20CU08nlixfZv3MX1erVTXemilWrcOjgQY4cPkzk1av8Pms2je6L3yPT8L4mzJw2DYCFc+dR6557ALBYsmXNhp+fH7dkycLVK1c5d/YcQOw+Pz8/smTNytHw8HRnFRGR/xbrdM8jNcaYZsaYXcaYIGPMW8m0ecQYs90Ys80Y85Mr3m+KE1iNMa+mdNxaO9wVIfIHFuRoaFjs9rGwcMpWqZym1+7fuYvHur/AjHETuSVrFirXrsmhoL3pzhQQEEh4SGjsdnhoKJWqVk3QJiC2jdPp5NzZs9yaJw8L586jYdMmLF63hqxZszJ08HucOX2aM6dPM2HsNyz4cxWXLl1i9cpVrF65Kt1ZRUTkv8UTS3uNMX7Al0BTIBhYb4yZaa3dHqfNHUAfoI619qQxpqArzp3aapqcrjhJ6pK46Al6QJLz96rVlKlUkU+mfM/pEyfZuXETTqcL7haTZKQEmUziRtZaKlapQpQziia17iZX7tyMnzKZNav+4Mzp0zRq2oTm9Rpw9swZPh41kpYPPsCcX39Lf14REZH0qQkEWWv3ARhjJgEPANvjtHkO+NJaexLAWhvhihOnWIxYawfdzA81xnQFugJ8/fXXqbY/FhZOgUKBsdv5AwM4HpH29zdp9FgmjR4LQO/hHxFy4NANJk4sPCyMgMKFYrcDChXiaIJM19qEh4Xh5+dHjpw5OX3qFC0eaMMfy5cTGRnJiePH2fjXX1SoXAlrLcGHgzl54gQAi3+fT9Vq1VSMiIhIPFGeuQNrEeBwnO1goFaCNmUAjDF/AH7AO9ba39N74hTnjBhjRqT0SO511tox1trq1trqXbt2TTXE7i1bKVyiOAFFi+CfyZ8GLZuzZvGytL0Bh4Oct+YGoETZMpQsewd/rfozTa9NybZNmyleogRFihbFP1MmmrVuxbKFi+K1WbZwMW3atQOgaYvmrPtzNQChISHUjJk/kjVrVirfVZX9e/cRFhJC5buqkiVLFgBq1bmHfUFB6c4qIiKSFsaYrsaYDXEecT+kkxobSjhM4Q/cATQEOgLfGGNuTW+u1IZpnge2AlOAEJIOmm5RTiejB33Ae+O+ws/PjwVTZ3AoaC+der7E7i3bWLtkGWUqVWDAqM/JkSsntRo14ImXX+T5Fg/h5+/Pxz9PAODCuXMMe70PUS4YpnE6nXzw9juMnjgBPz8Hv075hb179vBir1fYvmULyxYtZsaUyXwwfDizly3h9KnT9O7xMgCTJn7P4GFDmb7gd4wx/PbLVPbs3AnAonm/M3nOLJyRkezYtp2pP09Kd1YREflvcdecEWvtGGBMMoeDgWJxtosS/dmfsM0aa+1VYL8xZhfRxcn69OQyieZBxD1oTD7gYaADEAlMBqZdGytKI9v8jkrpyeg28/ZsoXKJUqk39JDNB/Z5OoKIiETL0BmlW5rVTtvEyRtU6fc1yb4PY4w/sBtoDBwhusB4zFq7LU6bZkBHa21nY0x+YCNQ1Vp7PD25UhymsdYet9Z+Za1tBHQBbgW2GWM6peekIiIi4l2stZFAd2A+sAOYYq3dZox51xjTJqbZfOC4MWY7sBR4I72FCKTxu2mMMf8jemyoKTAP+Cu9JxYREZGkeeq7aay1c4G5Cfa9Hee5BV6NebhMavcZGQS0IrpCmgT0iamcRERERFwitZ6RAcA+oErM4wMTfW8NQ3SBlLY7k4mIiEiaeeKmZ56UWjFSMkNSiIiIiM9K7aZnBxPui5k9e9ymtAxHREREJI1Su+lZbWPMMmPMdGPMXcaYrUTfdyQ8ZnmPiIiIuFhUlHHLw1ulNkwzEugL5AaWAM2ttWuMMXcCPwPpvgWsiIiI+LbUihF/a+0CAGPMu9baNQDW2p0miS+JExERkfRzenEvhjukOEwDxF3pfDHBMc0ZERERkXRLrWekijHmDNFLebPGPCdmO4tbk4mIiPgob57f4Q6prabxy6ggIiIiEi3K+lYxktowjYiIiIhbpem7aURERCTjeOq7aTxFPSMiIiLiURnSMzJvz5aMOM1N2Xxgn6cjiIiIxOP0sTkjGVKM9K/cKCNOc8Pe27yUx8vV8HSMZP24Yz2fV2nq6RhJ6rlpoacjiIj8Z/naahoN04iIiIhHqRgRERERj1IxIiIiIh6lpb0iIiJeRhNYRURExKN0B1YRERGRDKSeERERES/ja8M06hkRERERj1LPiIiIiJdxWk8nyFjqGRERERGPUs+IiIiIl/G11TQqRkRERLyMJrCKiIiIZCD1jIiIiHgZTWD1oDvq1KDnzAn0mv0D9Z/umOh4jYdb033at7w0ZSzPjR9BgVLF4x3PHViQAWvmUqfzIy7PVrnu3QybO5VPfp9O62c7JzrevPNjDJ01mSG//kSfcaPIXzgw3vGs2bPzxbI5dO7/hsuzJVT8nuo8+ds4Os8aT/WnOyTbrnSTevTctJCC5cu4PZOIiEhyvKYYMQ4Hrfv2ZOILbzHiwS5Uat44UbGxee5iRrZ7hi8feY6V4yfR/I0X4x1v0fsl9qxa65ZsXQb0ZmjXnvRu/Qh3t7yPIreXjNfm4I5d9H/4Sfo8+BjrFiym4+svxzve/uXn2bn+b5dnSyprw749+PXFvnz/0LOUadaIvKVuS9QuU7asVH3sQUI373B7JhERkZSkWowYYx40xrxujLnfnUGKVryT44dCOHkkFGdkJFt+X0K5RnXitbl8/kLs88xZs4C93o9VrlEdTgSHELH3gMuz3V65AuGHDnM0+AjOq5GsmbuQavc2iNdm+7q/uHLpMgBBm7aQN6Bg7LES5e8kd/68bPnD9YVSQgEVy3L6cAhnjoQRFRnJ7t+XUarhPYna3f1SF/4aPwXn5StuzyQiIjfGiXHLw1ulWIwYY0YBvYB8wGBjzAB3BckVkJ/T4RGx22fCj5KrYP5E7Wp1eJBX5/zA/b26MefDLwDIlDUL9Z7uyNLRE9ySLW/BAhwPC4/dPhEeTp6AAsm2b9juATat/BMAYwyPv/kKPw0b4ZZsCeUomJ+zYUdjt89FHCNHQPzrWODO28kZWID9K9xfHImIiKQmtZ6R+sC91to+QEPgwbT8UGNMV2PMBmPMhjFjxqQxSuKKzdrEM3jWTv6V4S2fYP5nY2jYtRMAjV/swp/fT+XKxUtpPNcNMmnLBlCndXNKVSzH7G+/B6BJx/ZsWvEHJ+IUM26VWlZjqP/6C6z45OuMySMiIjfMad3z8Fapraa5Yq11AlhrLxiTxCddEqy1Y4BrVYjtP/LnVF9zJvwoueMMbeQKKMDZo8eTbb9l3hLa9HsFgKKVylGhSQPu79WNLDlzYG0UkZevsHbSr2mJm6oT4RHkCwyI3c4bEMCpiGOJ2lW4uyYPdHuK957sRuTVqwDcUbUyZatVpUnH9mTJlg3/TP5cunCRycNHuiRbQufCj5Iz8HqvTY6C+Tkfcf06Zs6elXylS9D+m48ByJY/L60/f5dZPd8mYvtut2QSERFJSWrFyJ3GmM0xzw1we8y2Aay1trKrghzZtpN8xYuQp0ggZ8KPUanZvfzy1nvx2uS7rQjHDx0BoEz92rHPv+nSM7bNvS905vKFiy4rRAD2bdlOYPHbKFCkMCciIqjdoilfvhF/xKp4uTI8804fPur6MmdOnIzdP6r39Xb1H2xFyYrl3FaIAIRv28WttxUhV5FAzoUfo0yzhvzeZ0js8SvnLjCmYfvY7XbffMzK4WNUiIiIeBGnpwNksNSKkXIZkgKIckYx+4MRdB49FIefg79+nUfE3gM0fvEpjmzfxc5lf1Kr40PcXqsaUZGRXDxzlmn9P8ygbE7GvzeUN78ZgcPhx/LpMzkStI92Pbqxf+sO/l66gsfe6EmWbFnp+Wl0pmOhYQx/6bUMyReXdUaxbMhIHhw9BONwsP3X+ZzYe5DaL3YmfNtu9i9fneGZRETkxvhaMWKSm/uQ7AuMyQ8ct2l/oe1fudENB8sI721eyuPlang6RrJ+3LGez6s09XSMJPXctNDTEUREMlKGLkUZX7WpW2Z4dPlnoVcuqUltNU1tY8wyY8x0Y8xdxpitwFYg3BjTLGMiioiI+BZfW9qb2jDNSKAvkBtYAjS31q4xxtwJ/Az87uZ8IiIi8h+XWjHib61dAGCMeddauwbAWrszjQtrRERE5AY5b3AKxb9davcZiYrz/GKCY751pURERMQtUusZqWKMOUP0xJ2sMc+J2c7i1mQiIiI+ytdW06RYjFhr/TIqiIiIiPgmr/nWXhEREfFNqQ3TiIiISAbztWEa9YyIiIiIR6lnRERExMv4Ws+IihEREREv4/Sxu2domEZEREQ8Sj0jIiIiXsbXhmnUMyIiIiIeZaz773/vWwNfIiLyX5ShX8j2YeXGbvnsfGvzYq/8YjkN04iIiHgZXxumyZBipEv5Whlxmhs2fvtaShcv7ukYyQo6eJBO5Wt6OkaSvt++DoDHy9XwcJKk/bhjvacjiIhIGqlnRERExMtoaa+IiIhIBlIxIiIiIh6lYRoREREvo2EaERERkQyknhEREREv42tLe9UzIiIiIh6lnhEREREv43T/3dG9inpGRERExKPUMyIiIuJlfG01jYoRERERL+NrxYiGaURERAQAY0wzY8wuY0yQMeatJI4/b4zZYoz5xxizyhhT3hXnVc+IiIiIl4nywARWY4wf8CXQFAgG1htjZlprt8dp9pO19quY9m2A4UCz9J7ba4qRSnVr81ifV3H4OVgxdSZzvpkY7/j9nTtSv/0DREVGcvbkKb7t/x7HQ8IAyFsogKff7UfewIJY4NNuvTgWEpruTPUbNKD/wIH4+fkxZdIkvh49Ot7xzJkzM2z4cCpWqsTJkyfp2b07R4KDqVylCu8NGQKAMYYRn33GwvnzAVi2ahXnz5/H6XTidDp5qHXrdOeE6OvXqc9rOPwcLJv6G7MTXL9mnR+jYfs2OCOdnD15irH9B8devwlbVnN4z14AjoeE8Wn3112S6ZrKde+mU9/XcDiis836ZkK84807P0aj9g/gdDo5c+IUY/u/y7GYbABZs2dn6JwpbFi0jAnvDXNpNhERiVUTCLLW7gMwxkwCHgBiixFr7Zk47bODa8aTvKIYMQ4Hnfq/wbBne3AiPIKBk8ezcelKQvbuj21zcMduBj3cmSuXLtOoQ1seea07o1/rD0DXIQOZ9fV4tq1exy3ZsmKjotKdyeFw8M7gwXR+/HHCwsKYPnMmixctImjPntg2D3fowOnTp2ncoAEtW7em91tv0bN7d3bv2sVDrVvjdDopULAgs+fNY8miRTid0bexeeLRRzl58mS6M15jHA469+/NR89250R4BO9OnsDfia7fLt6OuX6NO7Tj0dd68OVr/QC4cvky/ds+4bI8CbN1GdCbIc9050R4OIOnTODvpSs4kiBb/4efjM72aDs6vv4yX7zaN/Z4+5efZ+f6v92ST0TEG3lozkgR4HCc7WCgVsJGxpiXgFeBzMC9rjixV8wZKVWpPOGHgjkaHILzaiRr5y3krnvrx2uzc91fXLl0GYC9m7eSN6AgAIVvL4nDz59tq9cBcPnCxdh26VGlalUOHjjA4cOHuXr1KnNmzaJJ06bx2jRp2pQZ06YB8Pvcudxdpw4Aly5dii08brnlFqybu9tur1Qh3vVbM28B1RJcvx1xrl/Q5i2x18/dbq9cgfBDhzkafCQ629yFVLu3Qbw22+Nm2xQ/W4nyd5I7f162/LE2Q/KKiPyXGWO6GmM2xHl0jXs4iZck+gCz1n5prb0deBPo74pcXlGM5AkoyImw8Njtk2ER5ClYINn29du2YfPK1QAElijGhbNn6f75hwyaNpEOr/fAONL/tgICAwkNvT7UExYaSkBgYOI2ISEAOJ1Ozp09S548eYDoYmbewoXMmT+fAf36xRYnFhj/ww/8Ons2HTp2THdOgDwBBeJdvxOpXL8Gca4fQKbMmRk0ZQIDf/6Wao0bJPu6m5G3YAGOx80WHk6egOSzNWz3AJtW/glED3E9/uYr/DRshEsziYj4KmvtGGtt9TiPMXEOBwPF4mwXBUJS+HGTgAddkeumh2mMMWOstV1Tb5mWn5XU3qR7E+5u3YySFcsx5MnnAXD4+VOmWlUGtuvE8dBwXvzkfeo92JIV02elL1NSiRL0cJgkgl9rs+mff2jetCm3ly7N0E8+YfmyZVy5fJkObdsSERFB3nz5mPDDD+zbu5f169alL2tSOZJpe0/M9Xs/5voBvNK4DaeOHqNA0cL0+W4Uh3cHEXH4SLoyxQmXOFsyPUV1WjenVMVyDO7UDYAmHduzacUf8QotERFf4KFhmvXAHcaYksAR4FHgsbgNjDF3WGuvzVdoCezBBVIsRowxeZM7BLRI4XVdga4AX3/9daohToRFkDcwIHY7T2BBTkYcS9Su/N01aN21C0M6v0Dk1atAdC/KoR27OBocXbz9vXg5t1epCOksRsLCwihUqFDsdmChQkSEx/9QDAsNpVDhwoSFheHn50eOnDk5depUvDZ7g4K4ePEiZcqUYeuWLURERES/5+PHWTh/PpWrVk13MZLw+uUNLMipiKOJ2lW4uwZtuj7FB52fj71+AKeORl/ro8Eh7Fz3N8XLlXVZMXIiPIJ8cbMFBHAqiX/bCnfX5IFuT/Hek91is91RtTJlq1WlScf2ZMmWDf9M/ly6cJHJw0e6JJuIiFxnrY00xnQH5gN+wDhr7TZjzLvABmvtTKC7MaYJcBU4CXR2xblT6xk5ChwkfkeBjdlOdtJBTLfPta4f++dn36Z4kv1bdxBQvBj5ixTiZMRRajVvyle9B8Rrc1u5MnQZ+BafdHuFsyeuT/7ct3U72XLlImeeWzl78hTlalfnwNYdqbyt1G3etIniJUtStFgxwsPCaNm6Na++/HK8NosXLeKhdu3Y+PffNGvRgjV/Rg8vFC1WjNCQEJxOJ4WLFKFkqVIcCQ4ma9asOBwOzp8/T9asWalbvz4jP/883Vn3bd1OYPFiFChSmBMREdRufh+jEly/4uXK8NTAPgzr1pMzca5ftlw5uXLxEpFXr5Lj1tzc8b/KzB73fbozxWbbsp3A4rddz9aiKV++kTjbM+/04aOuL8fLFvc91H+wFSUrllMhIiI+wVPfTWOtnQvMTbDv7TjPe7rjvKkVI/uAxtbaQwkPGGMOJ9H+pkQ5nfzw/se8PnYEDoeDlTNmERK0n4e6d2X/th38s3QlHV7vwS3ZsvHSpx8A0UtQP+/+BjYqisnDRtB73EgwhoPbdrJs6q/pzuR0Ohn09tt8N3Eifn5+/DJlCnv27KHnq6+ydfNmFi9axJTJk/nk009ZvHw5p06d4pXu3eYnXWMAACAASURBVAGoXr063V58katXr2KtZWD//pw8eZJixYoxakx0jebv78/M335jxfLl6c4a5XQy8f1hvBFz/VbMmMWRoH20jbl+G5eu5NHXXyZLtqz0+DR6yfG1JbxFSpXgqXf6YKMsxmGYPXZivFU4rsg2/r2hvPnNCBwOP5ZPn8mRoH2069GN/Vt38PfSFTz2Rk+yZMtKz08/BOBYaBjDX3rNZRlERP5tfO0OrCallR4xy3dWWWs3JXGsh7X2izScw3Ypn2hlkFcYv30tpYsX93SMZAUdPEin8jU9HSNJ32+PHlp6vFwNDydJ2o871ns6goj8tyQ5u9Fdnqtwt1uqkbHbVmfo+0ir1HpG1gGxEyWMMU8C7YgeunnHfbFERER8lyfuwOpJqa2B/Rq4AmCMqQ98CEwETnN9ToiIiIjITUutZ8TPWnsi5nkHYIy1dhowzRjzj3ujiYiI+CZfmzOSWs+InzHmWsHSGFgS55hX3EpeRERE/t1SKyh+BpYbY44BF4GVAMaY0kQP1YiIiIiL+VrPSIrFiLX2fWPMYqAQsMBeX3rjAHq4O5yIiIj896U61GKtXZPEvt3uiSMiIiK+RvM+REREvIyW9oqIiIhkIPWMiIiIeBlNYBURERGP8tQX5XmKhmlERETEo9QzIiIi4mWifGyYRj0jIiIi4lHqGREREfEyvjZnxFj3v2HfuqIiIvJfZDLyZG3KVHXLZ+fM3f9k6PtIqwzpGWl35/8y4jQ3bNrOv6lb6k5Px0jWqn076VK+lqdjJGn89rUANL+jkoeTJG3eni1c/m2Ap2Mk6ZYHBns6goiIV9EwjYiIiJfxtfuMaAKriIiIeJSKEREREfEoFSMiIiLiUZozIiIi4mWibJSnI2QoFSMiIiJeRndgFREREclA6hkRERHxMr52B1b1jIiIiIhHqWdERETEy/janBEVIyIiIl4mSsM0IiIiIhlHPSMiIiJexrfuMqKeEREREfEwrylGqta9hxHzpjNy/m889FyXRMdbd3mcz2ZPZfhvkxn43VcUKFwIgBJ3luGDSeP5bNYvDP9tMvc0v88t+WrVr8tPi+Yxacl8nnj+uUTHq9Sozrczp7Fs91YaNr8/0fFsObIz48/l9HrHPV9rX6lubYbMmcJHv0+l5bNPJjp+f+eOvD9rEoNn/EDvcSPJVzgw9ljeQgG8PnYEH8yaxPuzJpE/5tq6SrV6dRg7fybfLprDw12fSXS8Yo1qfPHrZGbv2EjdZk3jHXv6jV6MnjOd0XOmU79F4uvqaqt2hdJ66DxafjSXb5fuSHQ85OR5nh2zjHbD5/P0V0sJO3XB7ZlExPdEWeuWh7fyimEah8PBc2+/ybtPv8jx8HA++uUH1i9ZTvDe/bFt9u/YRe/2T3Dl0iXuf7Q9nV7vyfBX3+LypUt88eYAQg8eJk/B/Ayb+iP/rPqTC2fPuTTfq4PepteTTxMRFs43v/7CqkVLOBC0N7ZNeEgoH/TuQ8dnn07yZzzXqyf/rFvvskxxGYeDTv3fYNizPTgRHsHAyePZuHQlIXGu38Eduxn0cGeuXLpMow5teeS17ox+rT8AXYcMZNbX49m2eh23ZMuKjXJdB6HD4eCld/rRt0tXjoWF8fm0SaxdspRDQfti20SEhPLJmwNo90zneK+t0bAet1cox0ttHiZT5swM/fE7NqxYxYVz512WLy5nVBQfzPibMc81ICB3Vjp+sYiG5Qtze0Du2DafzN5E6/+V4IHqJVgbFM6I37fwwaO13JJHRMRXeEXPSOnKFQk7FEx48BEir0ayau58ajRuGK/N1rUbuHLpEgC7N20hX2BBAEIPHCL04GEATkYc4/SJk+TOm8el+cpVqUzwwUOEHA4m8upVFs2eS92mjeO1CTtyhL07dxMVlbjyLFuxAnny52Pdyj9cmuuaUpXKE34omKPBITivRrJ23kLuurd+vDY71/3FlUuXAdi7eSt5A6KvX+HbS+Lw82fb6nUAXL5wMbadK5SpXImQg4cIOxxM5NVIls+ZR+3GjeK1iTgSwoFdu7EJqvbbSt/OlnUbiHI6uXzxIvt37qJavbouy5bQ1sMnuC1/Dormy0Emfz+aVbmNpdtC4rXZF3GGWqWjr13N2wuydNsRt+UREfEVKRYjxhg/Y0w3Y8xgY0ydBMf6uypE3oACHAsNi90+ERZBvpgPy6Q0bv8gf69I/MFeulIF/DNlIuxQsKuiAVAgMICI0NDY7aOhYRQICEjTa40xdO/7JqM+HObSTHHlCSjIibDw2O2TYRHkKVgg2fb127Zh88rVAASWKMaFs2fp/vmHDJo2kQ6v98A4XFej5g8syNE4/7bHwsLJl8Zrt3/nLqrXr8stWbKQK8+tVK5dkwKF0vbamxF++iIBubPFbgfkzkrEmYvx2pQpdCuLtkb/fi3eeoTzlyM5dd51xZuICETfZ8QdD2+V2qfO10AD4DgwwhgzPM6xtsm9yBjT1RizwRizYcyYMamGMJhE+xL+lXxN/dYtuL1CeX77dmK8/bcWyM/LQwczsu87yb72ZiVOl3y+hB564jFWL1tORJwPZFczSQVM5pfu7tbNKFmxHPPG/QCAw8+fMtWqMnnYCAY98hQFihah3oMtXZkuiWhpu3Z/r1rNhuUr+WTK97z56VB2btyE0+l0YbbUJUz/Wssq/LXvKI98toAN+45SMHdW/BxJ/gOIiEgapTZnpKa1tjKAMWYkMMoYMx3oSNKf0QBYa8cA16oQO3/4Vyme5Hh4BPkLxZlQGViQExFHE7WrfHdN2j3/DAM6PUvk1aux+7Nmz06/rz7n589GsWfTllTe0o2LCAunYKHrkzoLFArkWEREml5b8X9VqVKjGg898RhZs2UjU6ZMXLxwnq+GDk/9xWl0IiyCvIHXewzyBBbkZMSxRO3K312D1l27MKTzC7HX72RYBId27OJocPRwxN+Ll3N7lYowfZZLsh0LC6dAnH/b/IEBHE/jtQOYNHosk0aPBaD38I8IOXDIJbmSEpA7K+Gnr09IDT99kQK5ssZrUzB3Vj59MrqT8MLlqyzaGkzOrJndlklEfJM3TzZ1h9R6RmL/X9ZaG2mt7QpsApYAOVwVImjLNgoVL0bBIoXxz+RP3Rb3s2HJ8nhtSpYrS7dB/fjwxVc4c+Jk7H7/TP70HvkJy36bw+r5i1wVKZ6dm7dQrERxChUtgn+mTDRp1YI/Fi1J02vf7fUG7erey8P1G/PlkKH8PuM3lxYiAPu37iCgeDHyFymEXyZ/ajVvysalK+K1ua1cGboMfIvPu7/B2TjXb9/W7WTLlYuceW4FoFzt6vEmvqbX7i1bKVyiOAFFi+CfyZ8GLZuzZvGyNL3W4XCQ89boyaMlypahZNk7+GvVny7LllCFonk5eOwcwSfOcTXSye+bDtGwfOF4bU6evxw7L+ibpTt5qHpJt+UREfEVqfWMbDDGNLPW/n5th7V2kDHmCDDaVSGinE6+GfwRA779EofDwZJpMzkctI9HezxP0NbtbFi6giffeIUs2bLx2mdDATgWGsaHL/binmb3Ub76XeS8NTeNHmoNwMg+Azmwc7er4uF0Ohn+zmCGT/gWh8PBnF+msX9PEM+80oOdW7byx+Kl3Fm5Ih+MHknO3Lmo07gRz/TsTqdmrV2WISVRTic/vP8xr48dgcPhYOWMWYQE7eeh7l3Zv20H/yxdSYfXe3BLtmy89OkHABwPCePz7m9go6KYPGwEvceNBGM4uG0ny6b+6tJsowd9wHvjvsLPz48FU2dwKGgvnXq+xO4t21i7ZBllKlVgwKjPyZErJ7UaNeCJl1/k+RYP4efvz8c/TwDgwrlzDHu9D1FuHKbx93PQ94H/8cI3K3BGWR6sUZLSgbn5cv5WyhfNQ6MKRVi/N4IR87ZgDPyvZAH6PfQ/t+UREd/lzfM73MGkNPfBGFMDCLbWhsZsPwm0Aw4C71hrT6ThHLbdnd75f9jTdv5N3VJ3ejpGslbt20mX8t65bHT89rUANL+jkoeTJG3eni1c/s0993RJr1seGOzpCCJy4zJ0clj1kqXdUo1s2B/klZPc0jKB9TKAMaY+8CEwETjN9TkhIiIiIjcttWEavzi9Hx2AMdbaacA0Y8w/7o0mIiLim5K4ZdV/Wmo9I37GmGsFS2OiJ65e4xV3bxUREZF/t9QKip+B5caYY8BFYCWAMaY00UM1IiIi4mK+NoE1xWLEWvu+MWYxUAhYYK/PdnUAPdwdTkRExBepGEnAWrsmiX2uWzcrIiIiPs0rvihPREREfJeKEREREfEorYgRERHxMj721TQqRkRERLyNr01g1TCNiIiIeJR6RkRERLyMb/WLqGdEREREPEw9IyIiIl7G1+aMGOv+Kbu+dUVFROS/yGTkye4sXsItn507Dx7I0PeRVuoZERER8TK+9ld8hhQjpUqWy4jT3LB9+3fQovV3no6RrLmznqJ5i9GejpGkeXNfAKBSxcYeTpK0LVsXU7Z4CU/HSNKugwcAuKtKS88GScbGTXM8HUHE5/laMaIJrCIiIuJRGqYRERHxMr42gVU9IyIiIuJRKkZERETEozRMIyIi4mV8a5BGPSMiIiLiYeoZERER8TLqGRERERGfZIxpZozZZYwJMsa8lcTxW4wxk2OOrzXGlHDFeVWMiIiIeBnrpkdKjDF+wJdAc6A80NEYUz5Bs2eAk9ba0sCnwEfpeZ/XqBgRERHxMp4oRoCaQJC1dp+19gowCXggQZsHgAkxz6cCjY0x6f6+GxUjIiIiAlAEOBxnOzhmX5JtrLWRwGkgX3pPrGJERETERxhjuhpjNsR5dI17OImXJOxQSUubG6bVNCIiIj7CWjsGGJPM4WCgWJztokBIMm2CjTH+QG7gRHpzebwYeXtgXxo2rM+lS5d44/W+bNu2PVGbihXLM2zYEG7JcgvLlq3g3UEfADDii+GUKlUCgFy5cnHmzBlatWxLkSKFWbhoDvv27Qfgn42b6N9/ULqzdutaixrVinL5ciTDP1/F3r3HE7V5952m5M2bDT8/w7Zt4Yz6ag1RUZZSJfPS/cW7yZTZjyin5cvRq9m951i6M8X1fLc61KhRnMuXI/lk+BL27k388we/2zImn4Ot20IZNWolUVGWt95qStEitwKQI0dmzp27Qvcev7gs21t9XqJevVpcunSZ/v2GsmPHnkRterz8NG3aNCVXrpzUqtkqdn9gYEHe/+BNcubMjp+fH599OpaVK9fddJZ6DRrQb+DbOPz8+GXSZMaOjv/NyJkyZ2bo8OFUqFSRUydP0at7d44EB1OkaFHmLl7E/r37ANi0cSMD+/UjS5YsfD56FLfdVhxnlJOlixbzyUcumdOVSO83u1GnbnUuXbrMwAGfsnPn3njHs2S5haHD+lC0WCBRUVGsWL6OEZ+Pd0sWEXGndE/DuBnrgTuMMSWBI8CjwGMJ2swEOgOrgfbAEmvtv7tnpGHD+pQoUZx7GzWjatUqDH7vbdo+9GiidoPfG0jfvgPZuPEfxn33NQ0a1GP58pW83OPV2DZ9+/Xm7JlzsdsHDx6mVcu2LstavVpRihTOxbPdplG2bAG6v3A3vV6fnajdkI+WcfHiVQD69WlE3TolWLFyP08/VZ2fJv3Dhr+OUL1aUZ5+qjpv9f3dZflqVL+NwkVu5Zlnf+LOsgF0716fXr2mJ843ZAEXruXrdz/16t7O8hVBfPjhwtg2zz57NxfOX3FZtnr1alL8tqK0bPEklSuXo/+Anjz+WPdE7ZYvW83PP/3KnLkT4+3v1u1x5s9fxpTJsyhVqjijRn9As/sfv6ksDoeDtwe/y1OPP0F4WBhTZ85kyaKF7N0TFNvm4Q6PcOb0ae5r0JAWrVvz+ltv0at7dN5DBw/yYIsWiX7uuDFjWbt6NZkyZWL8Tz9Sv2FDVixbdlMZk1O3bnVuu60wD7R+jkqVytK3/0s8+cSridpNnDidDes34+/vz9dj36dOnWr88cdfLs0iIv891tpIY0x3YD7gB4yz1m4zxrwLbLDWzgS+Bb43xgQR3SOS+EP7Jnh0zkiTpvcyY/pvAPzzzyZy5cpFgQIF4rUpUKAAOXLkYOPGfwCYMf03mt7XONHPatGiGbNmzXFb1tq1b2PxkugPrF27jpI9e2by5MmaqN21QsTPz+Dv74gdSbMWsmXNDED27Jk4ceKCi/OVYPHiXQDs3BVOjuy3kCdPtkTtLsTmc5DJ34FNYqivfr3SLFselGj/zWrUqA4zZy4AYPPmHeTMmYP8+fMmard58w6OHUvc22ct5MieHYCcObNz9GjiHqm0qly1KgcPHCT48GGuXr3KnFmzaNz0vnht7m16HzOmTQNg/ty53F3nnhR/5qVLl1i7ejUAV69eZfvWbQQEBt50xuQ0aFSb2bOWALBlyy5y5sxO/vx5EmS5zIb1mwGIjIxk5469FAzI7/IsIuJuxk2PlFlr51pry1hrb7fWvh+z7+2YQgRr7SVr7cPW2tLW2prW2n2ueLceLUYCAwIIDQ2L3Q4LDSMwsGD8NoEFCQsNv94mLJzAgIB4bWrUrM7xY8c5cOBg7L5ixYowa/Y0fp40kRo1qqU7a/582Th67Hzs9rHj58mfL/GHPcDgQffx0w8duXjxKqv+PADAmLFrefrp6kwY9wjPPF2D8RNc+5dqvvzZOXb0es/QsWPnyJ8/e5Jt3xvckp9/6sKFi1dZtSr+71HFioU4eeoCISGnXZatYEB+wsKOxm6Hhx+9oQ/IUaMm0KpVYxYtmsSoUR8w5IMvbjpLQGAAYaHXh0DDQ0MJCAxI1CY0JLqN0+nk7Nmz5MkT/aFftFgxZsydw/eTJ1OtRo1EPz9nrlw0atKY1X/8cdMZk1OwYD7CwuNex2MULJj8JPYcObNTv0Et1q3d5PIsIiKulGIxYozJZozpbYx5wxiTxRjTxRgz0xgz1BiTI70nT2ppcsKhpyTbJPhrvk3rlsyM0yty9OhR6tZpTOtW7Xj/vQ/59LNh5MiR9AdzeiQ3SjZg4AKeeHIymTL5UaVyIQBatLiTsd+so/PTUxj7zTp6vlzXpVlMEhVvcsN4/QfM4fEnJkTnqxJ/1VbDBnewfJnrekUAklyBfgNDjC1a3Muvvy2gSZNHefHFvnwwpE+SvxdpypKG65Tc72VERASN7r6Hh1q05MPBg/lkxOdkz3H9PwM/Pz+GfzGC778bT/Dhw4l+RnolnT3ptn5+Dj78sDc//zSTI0fCkm4kIuIlUpszMp7o9cRZgTnADuBjoDUwGuiU1Itilgp1Bfj666/jHevU6TE6PNoegM2bt1Ko0PXu7MBCgYTH+csPIDQ0nMBC1/9yDQwMIDw8Inbbz8+P+5s1oU3r9rH7rly5ypUrpwDYunU7hw4dpmTJEmzZsi2VtxtfqxZ3cv/9ZQDYs+cYBeL0NOTPl53jKQy1XL3qZM3aw9SudRsb/wmhyb2l+XrMWgBWrjpAzx51bihLkvlaVaDZ/dE3x9u9J4L8Ba5/MObPn4Pjx1POt3bNAWrXLsHGjcEAOByGe+4pycsvT013tkcffYB27aPnVmzduovAwOvDbwEBBYiISPtQy0Ntm/P889F3Jd60aTu3ZM5Enjy5OXHi1A3nCgsLI7BQ4etZChUiIs7vE0T30BUqXJjwsDD8/PzImTMnp05Fn+vUlei5NNu2buXQwUOULFmSrVu2ADD4wyEc2L+fCePG3XCu5DzSoSVt2zaLPue23QQGxL2O+ZMdsur/dg8OHQrhpx9/c1kWEclIHpnA6jGpDdOUsda+BrwEVAB6WGtXAL2BKsm9yFo7xlpb3VpbvWvXrvGOff/9T7Rq2ZZWLduycMFiHmobfXO3qlWrcPbsWY4ejV+MHD16lPPnzlO1avTpHmr7AIsWLok9XqfO3ezdu5+wsOtDOXnz5sHhiH5rxYoVpUSJ4hw6FJzKW01s9tyd9Og5kx49Z7J6zSEa31sagLJlC3D+whVOnrwYr32WLP6x80gcDkON6kU5HBw93HH8xAUqVYwuvKpULsSRkDM3nCdRvtnb6N7jF7r3+IXVq/fTuHFZAO4sG8D585c5eTJ+MRKdL1tsvuo1biP48PUP9LvuKkpw8CmOHT9Pek2a9BsPt+/Gw+27sWTJH7RpEz0vo3Llcpw7dz7JuSHJCQuNoHat/wFQstRtZL4l800VIgBbNm2iRMkSFC1WlEyZMtGydWuWLFwYr82SRQt5qF07AO5v0YI1f/4JQJ68eWN/r4oWK0aJkiU4fOgQAK+8/ho5cubkg0Hv3lSu5EyZPIdHO/Tg0Q49WLp0Da1a3wtApUplY67jyUSvefGlTuTMkZ1hQ5NbvSci4l3StJrGWmuNMXOvLd+J2U73Up6lS5fTsFF9li6bz6WLl+jdu2/ssdlzpseuhhkwYBBDhw0hS5ZbWL58JcuWrYht16p1C2bNjD9xtWbN6rzS62Wczkiczij693+H06fTNwdi/YZgalQvyrdj2nH5spNPP18Ze+yLz9vQo+dMsmTxZ+CAxmTy98PhZ9i0KZS583YCMGLkH3R7rhZ+fg6uXnHyxcg/05UnUb71h6hRozjjvn2MS5cj+fTTpbHHRn7xMN17/EKWLJl4Z2BzMmXyw+EwbNp0hDlzr/cWNahfmmXLEy+5Ta+VK9ZSv14t5s77nksXL9F/wLDYY79M/ZqH23cDoNerXWnZ4l6yZLmFRYsmMW36XEaPmsiwYV/xzqBX6fRkO6y19O8/9KazOJ1O3n37bb6ZOBE/Pz+mTZlC0J49vPxqL7Zu3sKSRYuYOnkKwz4dzoLlyzh96hS9uvcAoEatmrz86qs4I504o5wM7NuP06dPExAYyAs9erA3KIgZc6J/F3+YOIGpkyan46oltmrleurWrc7M2d9w6dJl3nn709hjkyZ/waMdelCwYD6e6/oo+/Yd5udJIwCYPGkWM2YscGkWEXEz3+oYwaS0PNgY8w3wirX2XIL9twMTrLVpmfhgS5Usl76UbrJv/w5atP7O0zGSNXfWUzRvMTr1hh4wb+4LAFSqmHhlkzfYsnUxZYuX8HSMJO06eACAu6q09GyQZGzc5L5VaSL/YhlaHpQoUTrdf/An5cCBIK8sc1LrGfkayAGcAzDGPAm0Aw4BD7o3moiIiPiC1OaMfA1cATDG1Ac+BCYCp4Cv3BtNRETENxk3/c9bpdYz4metvTbTsAMwxlo7DZhmjPnHvdFERETEF6TWM+IX80U4AI2BJXGOefx7bURERP6TjHHPw0ulVlD8DCw3xhwDLgIrAYwxpQHX3aJTREREYnnzkIo7pFiMWGvfN8YsBgoBC+J8M58D6OHucCIiIvLfl+pQi7V2TRL7drsnjoiIiHj4q+MynG+9WxEREfE6KkZERETEo1SMiIiIiEdpea6IiIiXMV68DNcdVIyIiIh4G+NbAxe+9W5FRETE66hnRERExMsYH+sr8K13KyIiIl7HXL+pqtu4/QQiIiJulqEzSu8oXcUtn517gjZ55czYDBmmaVmmSkac5obN2b2JPpUaejpGsoZsWcboKk09HSNJL2xaCECn8jU9nCRp329fx5/16nk6RpLuWbkSgEOdqns4SdJu+34DT07529MxkjTxkf95OoJIxtAEVhEREZGMowmsIiIiXsaoZ0REREQk46gYEREREY/SMI2IiIiX0X1GRERERDKQekZERES8jCawioiIiGQg9YyIiIh4GWP8PB0hQ6lnRERERDxKPSMiIiJextfmjKgYERER8TK+Voz41rsVERERr6OeERERES/jaxNYvbYYqVbvHrr2exOHn4MFv8zglzHj4h1/8KlO3P/wQzgjnZw+eZLP+gzkaEio2/KUqVOTVm92x+Hnx/rpc1j+7U/xjtd8uA13d3yQKGcUVy5cZMagj4nYd5CiFe/koYGvA2AMLBo1nu1LVrk8X7F7qlP3zRcxDgc7Zsxj47jJSbYr1aQe93/yNlM7vsTR7bu5JXdO7v/kbQpWKMvOmQtYNWSky7NVqlubTn1ew+HnYNnU35j9zcR4x5t1foyG7dvgjHRy9uQpxvYfzPGQMADyFQrgmXf7kTcwALB83K0Xx1z873xrzZqU7NkTHA4iZs/myI8/xjteoHlzSrz4IleOHgUgdPp0ImbPjj3uly0bVX/4gRMrVrD/s89cmi1LpbvJ0+l1cDg4v+xXzsyekKhNtppNyN22K9Zarh7aw/HR/aNz5Qsg7zMD8M8bfe0iPu6J85jr/xt54q6iVAnMxWWnZey6Axw8dTH+e/B30K9RmdjtvNky8+fBE/z4T3DsvhpFb6XHPaUYuHAn+09ecHlGEfFuXlmMOBwOXhjYl/5PdeNYWDifTvuJNYuXcXjvvtg2+7bv5JW2j3H50iVadHyYp3v34qNXerslj3E4aNOvJ992fZ0zYUd5adJX7Fj6BxH7Dsa22TR3Eet+mQlAuYb30PKNl/juhd6EB+3ny0e7EeV0kjN/Xl6e+i07l68myul0ab56fXswq9ubnA8/RrufRnJg2WpO7jsUr12mbFmp9NiDhG/eEbvPeeUq674cT97SJclbuoTLMsXN1rl/bz56tjsnwiN4d/IE/l66kpC9+2PbHNyxi7cf7syVS5dp3KEdj77Wgy9f6wdAtyHvMPPr79i6et3/2bvv+Kaq/4/jr3OTTmhLSyeFsil7CEjZZSmggqgITlDZQ0BRQBQRRVQ2iggCrq+KyBKZslrKBmUW2rK7d+mgM8n9/dGSNjSlICmtP87z8eBB7r2fm7yT3NucnHvuDTb2dqgGg2UDKgp13nqL4EmTyE1IoPm335J88CBZ166ZlCXu2VNiQ6PGsGGknTpl2VwAQsF5yBTiPx+LPjkOz1k/kvnPfnTRha+d1qMGjk+9RuysN1Az01EcnY3Lqo6cRdrm1WSfO4qwsQPVwq8d0NzTEY/KNryz/Tx1XewZ2tqHj/aEmtRk6wx8sCvEOP1Rz4actI+w1QAAIABJREFUiLxhnLbVKvSq78alpJsWzydJ/1VyzEgF0KB5U6KvRxAbEYUuT8f+rTvw6+lvUnPm6HFysrMBCDl1FlcP9zLLU6NZQ5LCo0iJjEGv03F6+14adetoUpNzs/DbnLWdLSoqAHnZOcaGh9bG2jjfktyb+pIaEU16VCwGnY5LOwKo5d+hWN2jY4dy6vu16HJyjfN0WdnEngxGX2SeJdVt1oS48EgSIqPR5+k4sv0vWnfvYlJz4djf5GbnAHDpzFlcCt7LanVro2g0nDt8DICczCxjnaVUbtSIrKgocmJiUHU6EvfswaVTp7tev1KDBli7uHDj+HGL5gKwrtsEXVwE+oQo0OvIPPIX9q27mtRU7jaA9N1rUTPTATCkpQCgrVYbFA3Z544CoOZkoeZa9rUDeMTbiYPXkgG4nJyJvZUGJ9uSv+N4VLbB0VZLaGKGcd6zTauxLSSOPL3lG0uSJP033HNjRAgRVhZBiqrq4U5ibKxxOjE2nqoeHiXWPzZwACf2HyyzPI7ubqTGJhin0+IScPJwK1bnN/hpJm/7md5vjeLPOUuM82s0a8TEjd8xYcN3bJq1wKK9IgCV3F25WSTfzfhEKnm4mtS4NqxLZU83ru8/atHHLo2zhxvJsXHG6eTYeJzdi792t3R9ph9ngg4D4FXLh8z0DN5c/Dkfr/+JwZPHIxTLtp9t3NzIjY83TucmJGDt6lqsrqq/Py2+/x7fjz/G2r2g4SsEtcaN49rXX1s00y0aZ3f0yYWvnS45Ho2zaaNb6+mDlVdNPD5YhceH32HbrD0AVl4+qJnpuL75BZ4f/0yVwW9CGXzTcrGzJjmrsCGbnJWLi511ifV+Ps4cjUgxTtesYoeLvRWnYtIsnk2SpP+OO/51EkKkCyHSCv6lCyHSgbq35t9hvRFCiBNCiBMrVqy451BCiOIzVfM9Ct36PUH9po1Zv/L7e36cuw9kLk7xPEfWbGJe35fYsXA53Ue8YpwfcfYCiwa8xtLBI/Ef9hJa65L/WP+7fKW8XkLQYfJoDs1fbtnHvQvm3suS+oY6PNWb2k0bsXX1TwAoGg2+rVvy69zFfPj8UNyre9Pl6SfLMK15KQcP8vfAgZweOpQbJ05Q/733APAcMICUI0dMGjMWZeZtvX0/EIoGrUcN4j4dQeLX03EZ9j7CvjIoWmx8W5Hy62JiP3wVrXt1KnV5qmxy3h7xDr1/fjWcORKe35MigBdbVufXU1EPJJck/ZcIoSmTfxVVaWNGvgecgHdUVY0DEEJcVVW19p1WUlV1BXCrFaL+MW/pPYVKjI3D1dPTOO3q6U6SmT/4LTu0Y9DoYUx56Q10eXn39Bj3Ii0uASfPwm/zjh5upMUnllh/Zvtenn5/UrH5CVfDyc3KxqNebaLOh5pZ89+5GZdApSL5Krm7cjM+yThtXckOl3q16LdyHgD2ri70WTyL7RNmkHC+bDu6kmPjCwaf5nPxdOdGfEKxuibt29JvxGt8OmSU8b1Mjo3n+oVQEiKjAfh7TyD1WjQlcIPl8uUkJBT2dADWbm7kJpq+t7q0wnZ33J9/UnPUKAAcmjTBsUULPJ9+Go2dHcLKCn1WFuHLLdPo0yfHo3EpfO20Lu7ob5i+drrkeHIvnwW9Hn1CNLqY61h5+KBPjiP3emj+IR4g8+8AbOo15Wbg/efqUc8V/9r5vUdXUzILekLyx3u42FmTkmV+X6zhZIdGEVxLyR/gamulUN3Jjmnd6gPgZGvFxE51WHTgihzEKkkPmTs2RlRVHS+EaA38KoTYBHxFyV9sLSbsbDDetXzwqO5NUlwcXZ7ozdy3ppnU1GnUkHGzPmDGG2NITU4u0zyR50JxrVkdZ29P0uISadGnO2umfGJSU9XHm6Tw/D/8vl38SCy47eztSWpsAga9nipeHrjVqkFKdGyxx7gf8cGhVPHxxsHbk5txidTr7c/uaXOMy3MzMvne/znjdL+V8zi8YEWZN0QArpw7j2fNGrh5VyM5Ph6/Po/x9bsfmNTUbNSA1z6cxtyRE0hLTjFZt5KjIw7OVUhPuUFjvzZcPXfh9oe4LxkhIdhVr46Nlxe5CQm49uhB2EcfmdRYVa1KXlJ+486lY0eyrucPXL748cfGGrc+fajs62uxhghA7pXzWHnWQONWDX1yPPZ+j5H09fsmNVl/B2Df/nFuBm1BqeyE1tMHXUIUhpvpKJUcUByqYEi/gW3jNuRetcxrt+dSInsu5TfYWng50rOeG0ciUqjrYk9mnp7UbJ3Z9dr7OHM4vPD9zcozMPaPM8bpaf71WXM6SjZEJAl5am8xqqr+LYToCYwDAgHbsg5l0OtZNmsOH69ahqJR2LVuE+GXLvPym2O4eC6Yo3sDeWPKJGzt7Zm2ZC4ACdGxzBo9oczybP50Ma9/MxehUTixcTvxl6/Rc+xrRAWHciHgEO1fGEA9v9bodXqy0tL5fXp+Y6BWq2Z0feNF9Do9qsHAH7MXkXkj1aL5VL2BoDlf8eSyOQhFIWTTTlIuX6ftmCEkBIdxLfDwHdd/adtPWFe2R2NlRe1uHdgyamqxM3H+LYNez4+z5/LOt0tQFIX9G/8k6tIVnhk3gqvBFzi5L4jBk9/E1t6O8QvzX7Ok6FgWjpuMajDw69zFTF29FCEE14JD2Lduk0VyGen1XFm4kMbz5yMUhbitW8m6do0ab7xBRkgIKQcP4vXcc7h07Iiq16NLS+PSp59aNkNJDHqSf5yL+ztfgqLh5v7N5EVdwemZkeRevUDWyf1knz2MbTM/vD5bi2owcGPNEgwZ+dvXjV8X4z51GQhB7rULZOzbaPGIp2PSaOHlxNy+TcjVGVh5vPAMs497NTQ5i+bRGs7MD7pk8QyS9P+R8pCdTSPMjX0wLhSiLRChqmpswfQ04CNgGfCRqqp30yWhPtGghSWyWtzWsNNMa+Zf3jFKNOdsAMta9CrvGGaNPr0LgFcaP1rOScz76fwxDnXuXN4xzOoQFARA+CttyjmJeT4/neDVtf+Udwyzfnz+kfKOID28zI3iKjOtWjxRJkchTp7e+kCfx90qrem1HMgFEEJ0AcYDLwBpFI4JkSRJkiTJguQAVlOaIr0fg4AVqqquB9YLIcrgKk+SJEmSJD1sSm2MCCG0qqrqgB7AiHtYV5IkSZKkf6Ei92KUhdIaFL8CgUKIRCALCAIQQtQDLDsKU5IkSZIkQDZGTKiqOlsIsQfwAv5SC0e7KuSPH5EkSZIkSbovd3Nq7xEz88r+AhWSJEmS9JASysPVM/JwncgsSZIkSVKFIxsjkiRJkiSVK9kYkSRJkiSpXMnTcyVJkiSpglHk2TSSJEmSJJWnh+3UXnmYRpIkSZKkciV7RiRJkiSpgpE9I5IkSZIkSQ+QKLyoapkp8weQJEmSpDImHuSDdfAbViafnYeOrHygz+NuycM0kiRJklTByLNpysAbjf0exMPcs1Xnj9CpTsPyjlGiA1dC+KVlr/KOYdaLp3YBsLhFxcw34fQu9vl1Le8YZnU7EgjA2826lHMS8+af3U/ffj+Wdwyztm1+FYCwAW3LOYl5DTYeL+8IkvSfJHtGJEmSJKmCkb9NI0mSJEmS9ADJnhFJkiRJqmCEeLg+nmXPiCRJkiRJpRJCuAghdgkhLhb872ympqYQ4m8hxCkhRLAQYtTd3LdsjEiSJEmSdDemAntUVa0P7CmYvl0M0EFV1ZZAO2CqEKJaaXf8cPUDSZIkSdJ/QAW9Amt/wL/g9g9AADClaIGqqrlFJm24y04P2TMiSZIkSdLd8FBVNQag4H93c0VCiBpCiDNABPC5qqrRpd2x7BmRJEmSpAqmrAawCiFGACOKzFqhquqKIst3A55mVp1+t4+hqmoE0Lzg8MwmIcQ6VVXj7rSObIxIkiRJ0kOioOGx4g7Le5a0TAgRJ4TwUlU1RgjhBcSX8ljRQohgoDOw7k618jCNJEmSJFUwitCUyb/7tBkYUnB7CPDH7QVCiOpCCLuC285ARyC0tDuWPSOSJEmSVMEIpUJ+PH8GrBVCvAGEAwMBhBBtgFGqqg4DGgHzhRAq+T8uOE9V1bOl3XGFfLaSJEmSJFUsqqomAT3MzD8BDCu4vQtofq/3LRsjkiRJklTBPGxXYK0wz7ZpJz9emDYJoVEIWreZ7St/Mln+2JAX6PxcP/Q6PRkpKXz3/mySomPxffQRBk+daKzzql2T5ZM/4OSe/RbN165LJybMmI6iKGxZu47/ffOtyfIWbdvw5gfTqNvQl5kT3iZg+07jssCLwVwJDQMgLjqGqSPGWDQbgFeHNrR+dwxCUbi8cTvnv/vNbF2Nnp3pPG8GO14cS/L5MCpV8+CJDatIvx4JQOKZCxyfvdii2Wp2aEPXKfnZgjdu58Rq02zNBj5J80H9UPUG8rKy2DNrIclXwlG0WnrMmIh74waoBgOBX3xN1IkzFs0G4OL3KPUnjQdFIWbzVsJ/+sVkuecTvak7bjQ5CQkARK3bSMzmrVSuX48G776FtpI9qsHA9e9/In73Potm8+34KE9PeRNFo3B0w1b2rvrZZHn7gf3o+MIzGPR6cjOz+P2jucRduU6D9m3oO3EkWisrdHl5bJm/jEvH/rFoNnNGDm9L2zbe5OToWbDoIJevJJdYO2N6Nzw9KzNm/J9lksW+VXvc33gbFIXU3X+QsuGHYjWVO/Sk6uDhoELOtTBiF36ATa0GuI+agmJXGQx6ktZ9R8bBXWWSUZKkfBWiMSIUhZfen8z8YW+SEhfPB799x6l9QcRcvmasuX4hlICBQ8nNzsF/0DM89/Y4lr/9PqHH/uGjZ/J/VrySkyNzdvxO8MGjFs2nKApvfTSDSa++TnxsHCs3/c6B3Xu5dumysSYuOoZP353GC8NeL7Z+TnY2rz05wKKZihKKQptp49k7agpZcYk8/vNXRAYeJu1KuEmd1t4O3xeeJvHMBZP5GZHRbB90V1fs/VfZ/N8bz8aRU8iIS2TwL19xJeAwyUWyhW7by9nftwBQu2t7Ok8exR9j3qPps30B+Pm5Edi5VKH/0tmseXEcqKrlAioKDSZP5NSbb5MTn0Cb75aTGHSQzGvXTcrid+/l4nzTRpo+O5sLs2aTFRGFtWtV2nz/LclHjqPLyLBINKEoPDN9EstHvEVqbAIT16wgeN8B4q4UZvtn224O/74ZgCb+Hen3zji+Hf0ON1NSWT1uKmkJSXjWq82Ib+Yxq+ezFslVkjatvfGu5siwkZvw9XVl3Oh2THpnu9naDu19yM7WlV0YRcF9xLtEzRxHXlIcNb/4gZvH9pMbedVYYuVVA5dnhxIxbRiGm+lonPKvbG3IzSZ28UzyYiLQOLtSc95PZJ48jCHTMu+rJN2NCnrRszJTIc6mqdOsMfHhkSRGRqPP03Fs+y5ade9iUhN67B9ys3MAuHLmHM4exa+10vqxbpwNOmKss5RGLZoTeT2c6IhIdHl57N6yjU69TA+bxUZFcTkkDIPBgh+Ud6lqU18yIqK5GRWLQafj+s4Aqvt3KFbXfOxQzn+/Fn1urpl7KRseTX1JjYgmrSBb2I4A6tyWLfdmpvG2lZ2tsbHhUqcm4UdPApCVfIPc9Jt4NGlg0XyOjRuRFRlFdnQMqk5H3K69uHbpdFfrZkVEkhURlf8cEpPIS0nBytnJYtl8mjUiKTyK5MgY9DodJ7fvoUk302w5RV47aztbVPJfu6iQi6QlJAEQe+kqWhtrNFZWFstmjl+7GuzZl99ADw1NpFIla5yd7YrV2dpqGdC/Mb+utXwvl/Ex6jchLyaCvLgo0OlIO7CLSo92Nalx6vU0N7b/juFmOgD61BQA8qLDyYuJyJ+Xkog+NdnYUJGkB0UIbZn8q6gqRGOkiocbybGFpyunxMZTxd2txPpOzzzFuaDDxeY/2qcXR7f+ZfF8bp4exMfEGKcTYmJx8/C46/WtbWxY+cc6lq9fQ+dexcb+3Dc7d1duxiYYpzPjErF3dzWpcfati72HG9FBxXuNKnt70nvNMnqsnI9bq6YWzVbZ3ZX0Itky4hOp7OFarK75oH4M2fIDnSYNI/DzrwFIDLtMXf8OCI2Co7cn7o3q4+BR8nbxb9i4uZIdX7jt5cQnYONWPJ9bt660/d9qmnz6ETZmtk2Hxg0RVlZkRZZ6ocG75uTuyo0i+0VqXAJOZp5/x8EDmLbtV558azSb5iwptrx5r65EhVxEn5dnsWzmuFa1JyGhsHGUmJSJa1X7YnWvvNSSDZuCyckpu54RrYsbusTCayzpkuKwqmr62llX88G6mg81Pl1Jjc9WY9+qfbH7sa3fGKysyIuNLLOskiSVcphGCNFcVdUzBbetyL8G/aPAOeATVVUz77T+3RJC3HWt31O9qdW0EV+8OtpkvpNrVao3qEvwwSOWiGTCXDr1Hg4VPNupO0nx8VSrUZ3FP//A5dAwosMjLBiweEKTfELwyDujOTJjbrG6rIRkNvV+idzUdJwb1afLwplsfXY4upsWeWtLz1bgzG+bOfPbZnz7dKPt8BfZ9cFcgjftwKW2Dy/88jVpMXHEnD6PQa+3TK475LtdYtAh4v7ag5qXR7UB/Wg04z1OjZtkXG5d1YXGH07nwqw5lj2EZC6bmfs/uGYjB9dspFXfnvQc8Spr3v/UuMyjbi2emDSKFSPetlyue3D7e12ntjPVvBz4dtUJ3N0rld0D3812p9Fg7VWDiA9Goq3qQY3ZK7g+YbDxcIzGuSqeE2YRu2SmZd9XSZKKKa1n5Psitz8D6gHzATvgm5JWEkKMEEKcEEKcWLGixAu9GaXExuPiWXjYxdnTnRvxCcXqGrVvyxMjhvLl2HfQ3fYtr23vHvyzOxC9zsIfVkB8bBzuXl7GaTcvTxLj73jhORNJBbXREZGcPHKMBk0aWzRfVlwClTwLv/XZe7iSVdBFD2BVyQ6nurXosXIe/bb9hGuzRnRZNAuXxg0w5OWRm5rfTZ1y4SIZkTE41qxusWwZcQk4FMlW2d2Vm/FJJdaH7gigbreOAKh6A/vnfcMvg0axZeKHWDtU4kZ4lMWyQX5PiK174bZn4+5GTkKiSY0uLQ21YHuL/mMLDg0LDxVp7O1pvuBzrixfRVrweYtmS41LoEqR/cLJw43U+MQS609t30PT7p1M6l9bNJtf35tNkgV7bIp6sq8vXy56ki8XPUlychZuboU9Ia5V7UlKzjKpb9jQjXp1q/Ldt88w77PeeFdz5LPZj1k8ly4pHq1rYe+ltqoHuuTEYjUZx/aDXo8uPpq86HCsqvkAoNhVwnv6IhJ/WUZ22DmL55MkyVRpjZGiXy96AMNVVQ0E3gJalrSSqqorVFVto6pqmxEjRpRUZnT13AU8atbA1dsLjZWWR/v04tS+IJMan0YNePXDKXw57h3Sk1OK3cejTzzG0W2WP0QDEHLmLDVq1cSrujdaKyt6PtmXg7v33tW6Do6OWFnnH6t3cq5CszatuHbxkkXzJQWH4uDjTaVqnihaLTUf9ycqsPAwVl5GJhu6Pcfmvq+wue8rJJ69wP6JM0g+H4aNsxNCyd8MKnl74uDjTUZkTEkPdc/igkOp4uONo3d+tga9/bkSaHqIrYqPt/F27S7tjA0Ora0NWjtbAHz8HkHV600GvlpC+oUQ7GpUx9bLE6HV4tGrO4lBB01qrKu6GG+7du7IzYLBrUKrpdnnnxC7bScJewMsmgsg4lwIrjWr4+LthUarpVWfHgQHmGZz9SlsODbq0p7E8PzDCbYOlRm29HO2Ll7BtVNl92G6ZVso4yduYfzELRw+Gk6PbnUB8PV15WZmHikppo2RbdvDeOW1dbw2fAOTp+4gKjqNqdMtv99mXzyPlZcPWvdqoNXi2KkXN4+bnmGXcTQQu2atAVAcnLCq5pM/xkSrpdrUuaQFbCPj0B6LZ5OkuyEUbZn8q6hKS+YkhBhAfqPFRlXVPABVVdWCq6tZhEGv5+fZ85j07WIUReHAxi1EX7pK/3HDuRYcwul9QQycPB4be3tGL5wNQHJ0HF+OeweAqtW8cPF0J+z4SUtFMqHX61kw82MW/LAKRVHY+vt6rl68xBsTxxNy9hwH9+yjYfOmfLrsKxycHOnYoxtvTBjHK72foma9urwz+yNUgwGhKPzvm29NzsKxBFVv4MRnX9Ft2RyEonDlj52kXr5Os9FDSD4fZtIwuZ37I81oNmYIqk6PajBw/JPF5KalWzRbwJyveLog2/lNO0m+fB2/MUOICw7jauBhmg/uj49fKwx5erLT0/nrgy8AsHOpwoBlc1ANKhnxieyc/rnFchXm0xM2bxEtFs9DKAoxW7aRefUatYe/TlpICElBh6j+/LO4du6IqteTl5ZOyMefAeDesxtOrVqgdXLE84neAIR8/BkZFmpsGvR6Nny6iBHfzENoFI5t3Ebc5Ws8PvZ1IoNDCQ44SMcXnqGBX2v0Oh1Zaen8Oj3/EE2nF56hag1veo18lV4j8882WzHybTKSb1gkmznHT0TRtrU3q5YPICdHx8Ilh4zLvlz0JOMnbimzxy7GoCfh2y+o/uESUDSk7dlMbsQVqr4wkuxLF7h5fD+ZJw9TqWU7ai75DQwGEn9YjCE9FYeufbBr3ArFwQnH7k8CELfkI3KuhT24/NJDryIPNi0L4k5jH4QQ3902a6qqqnFCCE/gZ1VV72Y0pvpGY7/7yVhmVp0/Qqc6Dcs7RokOXAnhl5a9yjuGWS+eyr/uwuIWFTPfhNO72OfXtfTCctDtSCAAbzfrUkpl+Zh/dj99+/1Y3jHM2rY5v2EVNqBtOScxr8HG4+UdQSo7dz+40QJ6P76wTAYq7dg56YE+j7tVWtPrayBSVdUYACHEq0KIZ4HrFFyTXpIkSZIkC3vIekZKGzOyHMgBEEJ0IX8Q649AKnf4CWJJkiRJkqS7VVrTS6Oq6q3rOQ8CVqiquh5YL4Q4VbbRJEmSJOnhVJEHm5aF0npGNKJwFE0PoOgpJA/XKyVJkiRJUpkorUHxKxAohEgEsoAgACFEPfIP1UiSJEmSZGEP29k0d3y2qqrOFkLsAbyAv9TCU28UYHxZh5MkSZKkh9JDdpim1Gerqmqx66urqipPuJckSZIkySIerqaXJEmSJP0XCE15J3igKsSv9kqSJEmS9PCSjRFJkiRJksqVPEwjSZIkSRWMvM6IJEmSJEnSA/RwNb0kSZIk6b/gIbvOiOwZkSRJkiSpXD1cTS9JkiRJ+g9QH7IxI6LwoqplpswfQJIkSZLKmHiQD9Z34B9l8tm57ff+D/R53K0H0vR6s2mnB/Ew92zJuQP0a9CyvGOUaHPYKRa06FneMcx66/RuABa36FXOScybcHoX7zbrWt4xzPribCAAY5t2LOck5i09d5DOHceUdwyzgg5+DcCKCrrdjTi9i3WtKuY++9zJ3eUdQZJK9HD1A0mSJEnSf4Eir8AqSZIkSZL0wMieEUmSJEmqYFTZMyJJkiRJkvTgyJ4RSZIkSapgZM+IJEmSJEnSAyQbI5IkSZIklSt5mEaSJEmSKhp5mEaSJEmSJOnBkT0jkiRJklTBqMrD1VcgGyOSJEmSVMHIs2kkSZIkSZIeINkzIkmSJEkVjEHzcPUVPFzPVpIkSZKkCqfC9ow06tiOZ6ZOQNEoHF6/hd2r/meyvOPz/ek8+BkMBgM5mVn8NvMLYq9cK7M8j3TuwLDp76LRKPz1+0bWr/jOZHn/116m18ABGHR6UlNSWDJtJgnRMQDMXLmUBi2bc+Hvk3w88s0yyVerQ1v8p4xBURTObtzO8dVrTJY3H/gkLQf1x6DXk5eVza5ZC0i+Eo6i1dJzxkQ8G/uiGgzs++JrIk+ctmi2mh3a0HXKGISiELxxOydW/2ayvNnAJ2k+qB+q3kBeVhZ7Zi00ZusxYyLujRugGgwEfvE1USfOWDQbQIOOj9J/yniERuHYhq0ErPrFZLnfwH60f2EAql5PTmYW6z+aR/yV69Ro2pBnP5ycXyQEu77+nuC9QRbN1rhjO56bOhFFo3Bw/Z/sum0/6P7qIDo8+xQGvZ6M5Bv874NPSY6JA6D/pNE07dIBgO3Lv+efHXssmu2WCRMH4te+CTnZeXw6+0fCwiJMltvYWPHxJ8Op5u2KwWDg4IGzLP/mDwDGv/ksrR5pAICtjTVVnB3o23uyRXJV79CGDgXbXcjG7Zy+bbu7pXbPzvSaP4MNL4wl8XwY3n6P8OiEN9BYWaHPy+Powm+JPnbKIpmK8ujQlpbv5Oe7umk7od+tMVvn3bMz7ed+yJ6XxpByPgwAp/q1eeT9SWgr2YNBZc/LYzDk5lk8o1R+5ADWCkAoCgPff4ulwydxIzaeyb+t5Ny+AyaNjb+37uLg2vw/aE39OzLg3fEsG/V2meRRFIWRH05jxmujSIqNY/76nzm2J5CIy1eMNVfOh/DWMy+Rm51NnxcGMvTdicydOAWADat+wMbWlt6DnyuTfEJR6P7eeNaPnEJ6XAIv/bKUywGHSL4SbqwJ2baXM79vAaBO1/b4Tx7NhjHTaPZsXwB+fG44di5VeGbpp/z84lhQVYtl839vPBtHTiEjLpHBv3zFlYDDJtlCt+3lbEG22l3b03nyKP4Y8x5NC7L9/NwI7Fyq0H/pbNa8OM5i2W7lGzB9It+OeJvU2ATGr1nO+X0Hib9y3Vhzcttujvy+GYDG/h146p2xrBr9LrGXrrJk8EgMej0Ori5MWreaC4GHMOj1Fsv2/Ptv8+XwidyIjefd31Zy9rb9IOLCRT4f9AZ52Tl0HvQ0T789ltWTZ9CkS3tqNPZlznND0VpbMfH7pZwPOkz2zUyLZLvFr30Tqld354VBM2ncpBZvTx7MyBFzi9X9+utuTv4ThlarYdGSCbTza8zRI+f5csl6Y82zz/lTv351i+QSikKn98azdeQUbsYlMuCXr7gecJgbRbY7ACt7O5q++DRxZy4Y52XfSGXnmzPITEjCuV4t+i6bw8+9XrBILiPIC28eAAAgAElEQVRFodXU8QSNnkJmXAI9fl5KdOAh0m/Lp7W3o94LA0gqkk9oFNp+Mo3jH3xGatgVrJ0cMegss81JFcfD1hipkM+2ZrNGJIRHkhQZjV6n45/tu2nWvZNJTdE/qtZ2dqgW/IC6Xf3mTYm5HkFcRBS6PB1BW3fSrqe/Sc3ZoyfIzc4GIPTUGVw9PIzLzhw+RpaFPwSK8mzqy42IaFKjYjDodITsCKCuf0eTmtwij29lZ2t8varWqUnE0ZMAZCXfICc9A88mDSyWzaOpL6kR0aRFxWLQ6QjbEUAd/w53zHarseFSpybhRbLlpt/Ew4LZAGo0a0RieBTJkTHodTpOb99Lk26m21rO7dtawe287Bxjw0NrY42KZbfBWrftB39v30Pz7p1Nai4e/4e87BwArp4OpoqHGwBedWtz6fhJDHo9uVnZRIVepHEnP4vmA+jUqTk7dhwF4HzwNSo72FO1qqNJTU5OHif/yf9Gr9PpCQuNwN3Nudh99ejZht27T1gkl1vBdpdesN1d3hFArdu2O4A2Y4dy+vu16HNyjfOSQi6TmZAEQMqla2isrVGsrCyS6xaXpr5kRERzMyoGVacjYmcA1W7bZwGajBlK2Pe/YcgtzOfRvg2pF6+QGpb/ZSg3NQ0MBovmk6QH7Y6NESHEOCGEa8HtekKI/UKIG0KIo0KIZmUVqoq7Gzdi443TN+IScHJ3K1bXefAzzNj+G/3fHs36OYvKKg5VPdxJjI01TifGxlHVw73E+l4DB/D3/gNllud2ld1dSS/yemXEJ+DgUbVYXYtB/Xh9y490mTScfZ8vBSAh7Ap1/TsgNAqO3p64N2qAwx2e27/LllAkWyKVPVyL1TUf1I8hW36g06RhBH7+NQCJYZdvy1YfB4/i28H9cHJ3JbXIa5cal4CjmXztBz/NlG2/0PetUWyes9g4v0azRry18Xve2vAdG2YtsFivCOTvBykm+0E8VczsB7d0eOYpzgcdASAy9BKNO/thZWtDpSpONGj7CM6elntfb3Fzq0J8fIpxOiE+BVe3KiXWV65sR8eOzTjxd4jJfA8PF6p5VeWfv0MtkquSuys3i2x3N+MTqXTb+1q1YV0qe7oRvv9oifdTu2dnEkMuYciz7CEQO3dXsuIK39usuATs3Ez32Sq+9bDzdCcmyDRfZZ/qoKp0WvoZPX5ZRoMhz1s0mySVh9J6RkarqppYcHsxsFBV1SrAFOCbklYSQowQQpwQQpxYsWLFvacSotgscz0fQWs2MKvPIDYv+IbHRg6598e56zh3lwfAv19f6jVtzIaVP5RZnmLM5itedvq3zax+8lWCFq2k3fCXADi3aTvpcYm89MvX+L8zhpjTwRb9QL3b9/LMb5v54ckhHFy0krbDXwQgeNMOMuISeOGXr+nyzmhiTp+3bLYS8pnr4Di8ZhOf932RbQuX033Eq8b5EWcvsGDAUL4cPIpuw15Ca21dptlK2u7aPvkYPk0asvu7/PEuIYeOERx0mMn/W85rcz/i6ulg9JZ+7bi3fUOjUfhw5uusW7ePmOgkk2U9erYmIOAkBoOFepdKyyUE7SeP5vD85SXehXPdmrSbOIygj8vii46Z7c5ksaDF5NGcmV/8z6yi0eDaqinHpn9KwOsT8e7eCfdHW5VBRkl6cEobM1J0ubuqqhsBVFUNEEI4lLSSqqorgFutEPXNJT/eU6gbcfFUKfItroqHG2kJiSXW/7N9N89/8DY/39Oj3L3E2DhcPT2N066eHiTHJxSra9GhHQNHD+O9l95AZ+FvUneSEZeAQ5HXq7K7GxnxSSXWh+zYR4/pE9gJqHoDgfOWGZcN/mExKeGRFs5W+G2+srsrN++QLXRHAN2mT2AXc1H1BvbPK/xjPPCHRdwIj7JYNsjvCXEq8to5ebiRFl/ytnZ6+x4GvD+p2Pz4q9fJy8rGs15tIs9b5tv9jbh4k96MKh7upJrZD3z92tB7xBAWDh1rst3tXPEjO1fk73tDP/+QhOuWeV8HPNOFp/rlH1IIuXAdd/fCQy5u7s4kJaaaXe+dd18kMjKe39fuK7asR882LJxvfoDpv3EzLoFKRba7Su6uZBbZ7qwq2eFSrxZPrZwHgJ2rC48vnsXOCTNIPB9GJXdXei2cyb73vyA9MsZiuW7Jik/ArkgPpJ2HG1kJhfm0lexxrFuLrivnA2Bb1YUOi2ZxaOIMMuMTSPj7DLk30gCIPXCUKg3rE3/spMVzSuXHIMeMmFgnhPheCFEH2CiEmCiE8BFCvAaEl7LuvxZ+LgQ3nxq4eHuh0Wp5pE9Pzu47aFLj5lM40K1Jlw4kWPAD9HYXzwZTrZYPHtWrobXS0vmJxzm6J9Ckpk4jX8bMep9PRk0kNTmlhHsqG7HBoVTx8cbR2xNFq6Vhb3+uBB4yqani412YtUs7Y4NDa2uD1s4WAB+/RzDo9SaDS+9X3G3ZGvT250rg4RKz1e7SztjguD2bauFsAJHnQnCtWR1nb080Wi0t+nTnfIDptuZaJF/DLu1JKnjtnL09UTT5V0ms4uWBW60aJEfHYinXz4Xg7lOdqgX7Qes+PTi7z/TwX/WG9Xnhw3f5ZtwUMpJvGOcLRaGSU/7YjWoN6uLdoB4XDh2zSK6NG/bz+tA5vD50DkH7z9C7dzsAGjepRUZGFklJacXWGTb8KSpVtmPJ4nXFltXwccfBwZ5z564UW/ZvJQSH4uTjjUPBdle3tz/Xi2x3eRmZ/Oj/HL/2fYVf+75C/JkLxoaItUMlen/1CccXryLuVLDFMhWVEhxKZR9v7Kt5IrRaajzuT0xA4T6ry7jJn92fZfsTL7P9iZdJPnuBQxNnkHI+jLhDJ3CqXweNrQ1Co+DaugVpRQZcS9J/0R17RlRVnS6EGAr8CtQFbIARwCbgpbIKZdDrWffpAsYsX4CiUTiycSuxl6/Sd+wbhAeHcC7gIJ1ffBZfvzbodTqy0tL533uzyyoOBr2e5bM+Y+aqZSgahd3r/iDi0mVefHM0l86d59jeQIZOmYSdvT1TluSfSZAQHcPs0RMBmPPLaqrXqYWtvT2r9+/ky/dmcvLA4Ts95D1R9Qb2zfmSZ5d9hlAUzm3aQdLl63QYM4TY4DCuBB6m5eD++Y2NPB056Rns/OALAOxdqvDMss9QDQYy4pPYPv0zi+W6lS1gzlc8vWwOQlE4v2knyZev4zdmCHHBYVwNPEzzwf3x8WuFIU9Pdno6fxVks3OpwoBlc1ANKhnxieyc/rlFs0H+e/vHp4sY9s08FI3C8Y3biLt8jcfGvk5kcAjnAw7R4YVnqOfXGoNOR1ZaBr9NnwNA7VbN8X/jRQw6HapBZePshWTeMN8r8G+zrf10IWOXL0DRaDi8cQsxl6/yxNhhhAeHcDbgAAPeHouNvR3DFnwCQHJMHMvHT0Gj1TLpx/yxN9kZmfwwdZblD3EBhw+fw699E9as/Yjs7FzmfPqTcdnq76fx+tA5uLlVYcjQPly7Fsuq76YCsGF9IFv+zP/w7dmzLXssNHD1FlVv4OCcr+izbA6KohC6aScpl6/TeswQEoPDTBomt2syuD+OPtVoNeJlWo14GYBto6eSXaSxZ4l8pz7/ks5f5++z1/7YQdqV6zQePYSU82HE3CFfXnoGF/+3ju7/WwqqSuyBY8QeKHnci/TfpD5kFz0TdzoLRQjRFohQVTW2YHoI8CxwDZipqmryXTyG+mbTTqVXlYMl5w7Qr0HL8o5Ros1hp1jQomd5xzDrrdO7AVjcolc5JzFvwuldvNusa3nHMOuLs/m9amObFj97oiJYeu4gnTuOKe8YZgUdzG9graig292I07tY16pi7rPPndxd3hH+60oZ6GNZXSb8XSaniO5f3PqBPo+7VVrTazmQCyCE6ALMAX4AUikcEyJJkiRJkvSvlTaAVVOk92MQsEJV1fXAeiGE5S9JKEmSJEkSBk2F7MAoM6X1jGiEELcaLD2AvUWWVcirt0qSJEmS9N9SWoPiVyBQCJEIZAFBkH8BNPIP1UiSJEmSZGGq8nD1jJR2Ns1sIcQewAv4Sy0c7aoA48s6nCRJkiQ9jGRj5Daqqh4xMy+sbOJIkiRJkvSwkeM+JEmSJKmCUTXlneDBeriuqiJJkiRJUoUjGyOSJEmSJJUreZhGkiRJkiqYh20Aq+wZkSRJkiSpXMmeEUmSJEmqaB6yroKH7OlKkiRJklTRyJ4RSZIkSapo5Km9kiRJkiRJD44ovMJ7mSnzB5AkSZKkMvZAT2/xm3W6TD47j8xoUSFP03kgh2n61G/2IB7mnm2/eJYGNWuVd4wShV2/xtnefuUdw6xmO/J/JeCnlr3KOYl5r5zaRb8GLcs7hlmbw04B0LNek3JOYt7uS8H07fdjeccwa9vmVwH4voJud0NP7eJkz/blHcOsVrsPA7CgRc9yTmLeW6d3l3eEiqUCHrcQQrgAvwG1gGvA86qqppip8wFWAjXI75Doq6rqtTvddwV8upIkSZIkVUBTgT2qqtYH9hRMm/MjMFdV1UbAo0B8aXcsB7BKkiRJUgUjKmZXQX/Av+D2D0AAMKVogRCiMaBVVXUXgKqqGXdzxxXz6UqSJEmSVNF4qKoaA1Dwv7uZmgbADSHEBiHESSHEXCFEqecGyZ4RSZIkSapghFI2534IIUYAI4rMWqGq6ooiy3cDnmZWnX6XD6EFOgOtgHDyx5gMBVaVtpIkSZIkSQ+BgobHijssL3GEsxAiTgjhpapqjBDCC/NjQSKBk6qqXilYZxPgRymNEXmYRpIkSZKku7EZGFJwewjwh5ma44CzEMKtYLo7cL60O5aNEUmSJEmqYIRSNv/u02dALyHERaBXwTRCiDZCiJUAqqrqgcnAHiHEWfKvz/JtaXcsD9NIkiRJklQqVVWTgB5m5p8AhhWZ3gU0v5f7lo0RSZIkSapglIfst2lkY0SSJEmSKhjlIRtE8ZA9XUmSJEmSKhrZMyJJkiRJFUxZXWekopI9I5IkSZIklasK0zPSunNHRr0/BUWjYcfaDfy+wvT6KE3btmbk9Hep7duAzya9y4Edu4zLXn9nEm39OwPw69Ll7N+20yKZOnftyvQPZ6DRaPh9zW+sWLbMZLmVtTVzFyygSbOm3Ei5wcRx44iKjDQu96pWjW27d/HlokWsXvEt1jY2/LL2N6ytbdBoNezctp0lCxdaJGvl1n5UGz0JFIWUHZtJWPuTyfIqvZ7A641x5CUlAJD05zpSdmwGwPP1sTg82gGA+F++I3W/ZX89s1qHNrR5dwxCUbi0cTvB3/1mts6nZ2e6zpvB1hfHknw+zDjf3tONfhtWceabHzn/4zqLZgN4pHMHhk1/F41G4a/fN7J+xXcmy/u/9jK9Bg7AoNOTmpLCkmkzSYiOAWDmyqU0aNmcC3+f5OORb1o8W9sunRjz/lQUjYbta9ezZvlKk+XN2rZmzPtTqePbgE8mvkPQjr8AaOH3KKPfK/zJCJ+6tflkwmQO7d5r8Ywjh7elbRtvcnL0LFh0kMtXkovVzJrZAxdnOzQaheDgOL5efgyDQaV2LWfGjfHDzlZLXHwGX8w/QFZWnkVyeXdow6MF293Fjds5W8J2V7NnZ7rNm8GfL44l6XwYrk196fDBJOPyU9/8RPi+gxbJVJRDWz+qj5mIUDQkbd9M3BrTfdblsb5UGzGOvMT8fTbxj3Ukbf+Tyi0ewXv0BGOdrU9Nrn0yg9RD+y2WrVaHtvhPGYOiKJzduJ3jq9eYLG8+8ElaDuqPQa8nLyubXbMWkHwlHEWrodeHb+PRqD5Co3D+z90cX/2rxXI9TB62MSMlNkaEENuAMaX97K8lKIrC2JnTeW/oCBJjY1m8fg1H9+4j/NIVY018dAzzp3zAs28MMVm3rX9n6jZpxNh+A7GytuaLn7/jxP4DZGbcvO9MH348i9deepnY2FjWb97Mnt27uHzxkrFm4KDnSU1NpVdXf5546inemTqViePGGZe/N+MD9gcEGKdzc3J49YUXyczMRKvV8uu6dQQGBHD65Mn7yoqiUG3sZK6+9ya6xHjqLvmOtCNB5IRfMylL3b+b6K/nm8xzeLQDtvV8uTjmVYSVFXXmLiP9xCEMmZn3l6mAUBQenTae3aOmkBmXSJ+fvyIy8DCpV8JN6rT2djR84WkSzlwodh9tJo8m+uBxi+S5naIojPxwGjNeG0VSbBzz1//MsT2BRFwu3PaunA/hrWdeIjc7mz4vDGTouxOZOzH/g37Dqh+wsbWl9+DnyiTb+JnTmTJkOAmxcSzd8BuH9uwj/NJlY018dAxfvDud54cNNVn39JFjjOr3LAAOTk78sGc7fx84ZPGMbVp7413NkWEjN+Hr68q40e2Y9M72YnVzPt9vbGRMn9qVTh1rsj/oGhPGt2fl6r85FxxHr571eO6ZJvz086n7ziUUhXbTxvNXwXb35M9fEV7Cdtfotu0u5dI1/nxxDKregJ2rC/3WfkPE/sOoesN95zJSFGqMf5tLUyaQlxCP79LVpB4KIvu2ffZGwB4ivzLdZzNO/0PoqPy/gxoHRxr/8Dtpfx+1WDShKHR/bzzrR04hPS6Bl35ZyuWAQyQXee1Ctu3lzO9bAKjTtT3+k0ezYcw0GvTqisbaih+fG47W1oYhG1YRumMvadFxFssn/f90p7bX98BfQojpQgirsgzRoHkzoq+HExsRiS5PR+DW7fj16GZSEx8VzbXQMFTV9DiaT726nD12AoNeT05WFldDQmndudN9Z2resiXXr10nIiKCvLw8tv75Jz17PWZS06PXY2xcvx6AHdu20b5jB+Oyno89RkR4OJfCLpqsk1nwIa/VatFaaYs9n3/D3rcxuTGR5MVGo+p0pAbuwrF9l7ta18anNjfPngSDHjUnm+yrF3Fo3f6+M91Stakv6RHRZETFYtDpuL4zgBr+HYrVtRw7lODv16LPzTWZX6NbBzKiYrhx+ZrFMhVVv3lTYq5HEBcRhS5PR9DWnbTr6W9Sc/boCXKzswEIPXUGVw8P47Izh4+RddMyDbfb+bZoRvT1CGIiItHl5RGwdRsde5ruF3FR0VwNDcNgKHk76tL7MY4HBpFT8Bwsya9dDfbsy28chYYmUqmSNc7OdsXqbjVENBqBVqtAQdzq3o6cC87/oDp5KpqO7X0sksv1tu3u6s4AfMxsd4+MHcq527Y7fXaOseGhsbY2ZrUke9/G5ERHkhuTv8+mBOzGqePd7bNFVenSjbTjh1FzciyWzbOpLzciokmNisGg0xGyI4C6/h1NanKLbPNWdrbGv2OqqmJlZ4vQKGhtbDDodORmlM3+8f+dopTNv4qqxGiqqq4l/4duHIETQojJQoi3bv2zZAhXT3cSYmKN04mxcVQt8gf/Tq6GhNKmSydsbG1xdK5Cc79HcfO6u3XvxMPTg9iYaON0bEwMHp4exWpiovNr9Ho96enpODs7Y2dnx/DRo/hq0eJi96soCn9s28bhf/7mYNABzpy6/2+B2qpu5CUU/kRAXmI8VlXditU5dupGvWX/w2f6p1i55v/YYvaVizi0aY+wsUHj6ETl5q2xcrv/1+8We3dXbsYmGKdvxiVi5+5qUuPsWxd7Dzeigky/3WltbWkydBBnvjHtvrakqh7uJMbevu2Z+yHKfL0GDuDv/QfKLE9Rrh4exMfEGKcT7mG/KMr/yT7s3bLNktGMXKvak5BQ+GGTmJSJa1V7s7Ufz+zJLz89T1aWjgOHrgNw7foN/NrVAKBzx5q4ulaySC5z2539bdudS8F2FxlUvFfBtWlD+q//lv7rVnD4k8WW7RUBrF3dyI0v3GdzE8zvs1U6+9NwxU/UmjEbK7fi26Wzf09S9u4qNv9+VHZ3JT22MFtGfAIOHlWL1bUY1I/Xt/xIl0nD2ff5UgAu7t5PXlY2I3evZfjOnznxw+9kp6VbNN/DQjZGTOUBNwEbwOG2fxYkis+6yx6Dfw4c5kRgEPPX/sSUhV8QcvI0er3eAomKZ7q9F0MI8zVvvjWJ71euMvaCFGUwGOjfty9d/NrTvGUL6jdocN9ZMZvDdDr9SBChQwZwafTLZJw8TvXJMwDI+OcY6ccPUXfBt9SY+jGZF86h6nX3n+kO2UzCCUGbd0bz94Llxcqaj36VCz+vR5dl+W/0hQ9f+vt8i3+/vtRr2pgNK38oszxFmXvp7na/uMXFzZXavvU5EWT5MQ8lKen1+2Dmbl4e8jtWVgotmuf/KOiiJYd4sq8vixc8gZ2dFTqdhT7072K7e/Sd0Zwws90BJJ4L4Y9nh7PlpXE0e2MwGmsLdw6bf3NNplKPHCD45WcIGfEK6f+coOa7H5gs17pUxbZ2XdJOHCnzbObe0tO/bWb1k68StGgl7Ya/BIBn04aoegMreg1iZd9XaP3qczh5e1k2n/T/0p3GjPQGFpD/wziPqKp6131tRX+iePly8zt7UYmxcbh5Ff5isaunB0nx5n4M0Lw1y75lzbL8S9+/u+Bzoq+Fl7JG6WJjY/H0qmac9vTyIj7ONFNsTCxe1aoRFxuLRqPBwcGBGzdu0KJlSx7v05d3pk3D0dERg2ogNyeH//3wo3Hd9LQ0jh0+Qmf/rlwMC+N+6BLjTb41Wbm6o0tOMKnRp6cZbyfv+APPN8YapxPWfE/Cmu8BqDHlI3KjI+4rT1GZcQlU8iz8xlfJw5WshKTCrJXsqFK3Fo+tnAeAXVUXui2axb6JM3Bt1pCavTrzyMThWDtURjUY0OfkEfqbud9m+ncSY+Nw9TTd9pLjE4rVtejQjoGjh/HeS2+gy7PMAMvSJMTG4e5V+Ifc7R73C4CufXtz8K896HWWa2A+2deXxx+rD8DFi0m4udlDwZAL16r2JCVnlbhuXp6BI8ci8GtXg5OnYoiMSuP9D/MHTHtXc6Btm+oWyWhuu8s0s931LrLd9Vg0iz0TZ5BUZPB06tVwdFnZVKlX22T+/cpNiMfavXCftXZzJy8p0aRGn1a4zyZt+wPv4WNMljt37UHqwUCwwJevojLiEnDwLMxW2d2NjPikEutDduyjx/QJ7AQa9unOtUPHMej0ZCXfIPpUMB5NGpAaFVPi+pIEd+4ZmQ4MVFV16r00RCD/J4pVVW2jqmqbESNGlFofdvYc1WrVxKO6N1orLV2f6MORPQF39ViKouBQxQmAWr4NqO1b3yID9c6ePk2t2rWoXqM6VlZWPPHUU+zZZdodunf3LgY8mz9IsHffvhw+lP+4Lw58nu6dOtG9Uyd+WL2ab5Yu5X8//IiziwsOjo4A2NjY0KFTR64UGYz4b2WGXsCmWg2sPLwQWi1OXXuRdiTIpEbrUtjN6ujXuXBwq6KgccjPZFu7Hra165H+97H7znRLUnAoDj7eVK7miaLVUvNxfyICDxuX52Vk8nu359jY9xU29n2FhLMX2DdxBsnnw/jr9beM8y/8vIFzq361aEME4OLZYKrV8sGjejW0Vlo6P/E4R/cEmtTUaeTLmFnv88moiaQmp1j08e8k9Mw5vGv64FndG62VFf5P9OXQnn33dB/dn+pr8UM0W7aFMn7iFsZP3MLho+H06FYXAF9fV25m5pGSYtoYsbXVGseRKIqgbevqRESmAuDkZAvkfxkf/Hxztu2wzAd+YnAojkW2u9pmtrs13Z5jXd9XWFew3d1qiFSu5onQ5P9prOTljlPNGmREx5b0UP9KZugFbLxrYO2Zv886+/ck9VDJ+6xT+87FBrc6d+9l8UM0ALHBoVTx8cbRO/+1a9jbnyuBpn9Tq/h4G2/X6dKOlPD8swjTY+Op8WjL/Px2tng1a0Ty1fv/cij9/1diz4iqqp0fVAiDXs+yjz7lk9XfoNFo+GvdRsIvXeaVCWMJOxvM0b0BNGjWhA++XkxlRwfadevKy2+OYVTfAWi0Wub9mt9tnpmRwdzJ0zBY4JuCXq9n1owZrPrxRzQaDevWruXSxYu8+dYkzp05y97du/n9t7XMXbiAXYEBpN64waRx4+94n+7u7ny+YD6KoqAoCtu3bCVgrwVOtTToif56HrVnL84/tfevLeRcv4r7K8PJuhhC+pEgqvZ/Hke/zqh6Pfr0NCLnfwyA0GipMy+/98qQeZOIL2aCwXLftFS9gWOffUWPZXPyT+39Yyepl6/TYvQQks6HEVnkA6I8GPR6ls/6jJmrlqFoFHav+4OIS5d58c3RXDp3nmN7Axk6ZRJ29vZMWTIXgIToGGaPngjAnF9WU71OLWzt7Vm9fydfvjeTkwcs85wMej1ffjSbz75bgaJR2PH7Rq5fvMyQCeMIOxfM4T378G3WlJnLFlPZ0ZH23f0ZMmEsw/r0B8DDuxpunp6cOVo2ZyIBHD8RRdvW3qxaPoCcHB0LlxR+aH256EnGT9yCra2WD9/vhpWVBkURnD4Ty7bt+Y0O/y61eLJvQwAOHg5n1+5LZh/nXql6A0c++4peRba7G5ev07Jgu4u4w3bn3qopzV4fhKrToxoMHJmzhJwbaSXW/ysGPZFfzqfuZ4sQikLSji1kX7+K55DhZIZdIO3wAdwGPI9T+06g16NLT+P6F58YV7f28MTKzYOMM/d5Jp4Zqt7Avjlf8uyyzxCKwrlNO0i6fJ0OY4YQGxzGlcDDtBzcHx+/RzDk6chJz2DnB18A/F979x97VV3Hcfz5khpBjlmhU4aTAoxSFxitHzZdhlNYsWgmqRNwM8vm2irRypUoubUssiCXacpgDmEISaWUbpGWbE3hi0AYQkpr/RKrFUYW+O6Pz/m249d77/dL+97z+Xz5vh7bHfee87nc1/d87j33cz6fzzmXnnvv5/ybFjJv3Z0IsfP+H7P/6WcGPeNwUPL8jm7QYJzN0Y+YOfmMbr/G/+XBp7dz6ikTcsdoa/e+Z9l+wbtyx2jpjI1pnHrl1PMyJ2ntsp6HmH3q1NwxWtqwO01anjHptMxJWnt4z05mzV7Rf8EMHtgwD4Dlhb7vFvQ8xNYZg3c22mCa9nBqgBFEGIgAAAcaSURBVC1524zMSVr7zLbBvb5RF7Sa6NM1s+7Y2pUv5wc+Nq3Rv2Oghlnby8zMzEpTzBVYzczMLBnh36YxMzMza457RszMzAoz3CawujFiZmZWmOHWGBlmf66ZmZmVxj0jZmZmhRkxzLoKhtmfa2ZmZqVxz4iZmVlhjiny0mTd454RMzMzy8qNETMzM8vKwzRmZmaF8QRWMzMzswa5Z8TMzKwwvuiZmZmZWYMU0fVfBhxePz1oZmZHo0ZPtp23ZktXvjtXXHRmkScNN9EYGVSSroyI7+bO0U7J+UrOBmXnKzkblJ2v5GxQdr6Ss0HZ+UrOZq80FIdprswdoB8l5ys5G5Sdr+RsUHa+krNB2flKzgZl5ys5m/UxFBsjZmZmdhRxY8TMzMyyGoqNkdLHAEvOV3I2KDtfydmg7HwlZ4Oy85WcDcrOV3I262PITWA1MzOzo8tQ7BkxMzOzo0hxjRFJcySFpCm1ZZMl/VDSXklPSPqppLOrdQskPSepp3Z7a5eyXS9pp6Qnq9d5p6RNkn5de+21VdlvSfpin+d+uxu5aq9xuMqwTdIWSe+plk+otuniWtmxkv4jaVn1eJGka7qZb4A5D1brfiXpO5Iaf4+2qeerJe2ptuPYpjMNIN891ftwh6S7JL26sHzfq+r7SUlrJR1bSrbauqWSDuTI1SmfpOWSnqntY6YWlE2Sbpa0W9IuSZ/Kka2WUZJ+LmlmbdlFkjbmzGUDEBFF3YA1wKPAourxa4DdwOxamdOBBdX9BcCyBnK9G9gMjKwejwXGAZuA6S3KjwF+A7wJeCPwDHBclzMeqN0/H/hZdX8CsBfYWlt/FdDTu+2ARcA1DdVxp5w7qvuvAh4BPtzw+69dPU+r8j0LjG0y0wDzzSJdlEnAKuCqwvKNqZVZAnyulGzV/enAyvp7s5R8wHLgwly5+sl2ObACOKZafkLOnFWG04FdpO+O1wJPAxNz5/Kt862o36apjpbOAt4HbCB9QV4KbI6IDb3lImIHsKPheCcB+yPixSrD/ipzy8IR8XdJ1wPLqkVfioi/NRG0Mgb4a+3xQWCXpOkR8Tgwl9TwG9dgplb65gQgIg5JegyY1HCelvUM/B7a13eDOuYDkPRLYHyGbNA+H5COXIFR5Lkyc7vP8AjgFuASYE6GXL2OaB/TsHbZrgIuiYiXquV/zhcxiYgdkn4AXEdqjKyIiL2ZY1k/Shum+RCwMSJ2A3+RdCZwGrCln+fN7TNMM6oL2X4CnFx1R94m6Zzauntqr31L78KIWAW8jnRUuLILmfoaVWV4CrgTWNxn/b3ARyWNBw5T+wJrWH85kTQaeD+wveFsneq5BB3zVcMzlwG5uqXb5pN0N/BHYAqwtKBsVwMbIuIPGTLVdarbm6vhkW9IGllQtomk/e/jkh6UNDlDtlZuJDUuZwJfzZzFBqC0xsjFpC9Mqn8v7ltA0vpqXHxdbfHqiJhaux0c7GARcQB4O+mqfs8BqyUtqFZfWnvthbWs44ETgXENjZEfrDJMAS4AVujlh1UbgfNI23V1A3na6ZRzoqQe4BfAjyLiwSaD9VPP2Q0g323AIxHxaIZ4HfNFxOWknrhdpJ65ErJ9AfgIeRpHL9Nh232e1IB7B/B60hF/KdlGAv+KiOnAHcBdTWdrJSJeIO3jVvb25ljZihmmkfQG4FzgdEkBjCB15d4InN1bLiLmSJoOfK3pjBFxmDRHZJOk7cD8fp7yTdJQ01uAG4CFHUsPoojYrDTR8vjasn9LegL4LKnH6YNN5WmnRc69EZFlgl4tU6t6Xp4zU127fJJuIG3Hj2eM13H7RcRhSatJn4W7C8i2Cnge2FO1h0dL2hMRTQ8Ptss3PyKWV6tfrHqXGploPpBswO+A+6oi68lQpx28VN1sCCipZ+RC0tjeKRExISJOJk363A2cJWl2rezopsNJenOfLsipwL4O5WcCJ5Amdy0G5qhLZ/m0ef0ppAbd831WfR24LiL6Ls+iQ84sjrSem9Yun6QrSJOBL+4dv8+hTb7fSppUrRepEfxUIdluj4gTq33OBOCfuRoiHer2pGq9SEPZTc+X6/S5+D7pIBLgHNL+2uyIFdMzQho6+EqfZfeRxv0+ACyRdCvwJ+AfwJdr5eZKem/t8Scj4rFBzncssFTSccAhYA+py3Itac5I79DQ/irvraQZ8AG8IOla0mTWc1/xPw+eUdUQB6SzKuZXR6L/KxARO4GdXcwwEP3mzKhlPSudsngtadjtSUkPRMQVpeQjzcXYB2yutuO6iLipkHyfANZLGkOq722ks7lKyFbSj6m1y7dG0vGkbddD2p6lZDtE2v99GjgA5PhM2FHAV2A1MzOzrEoapjEzM7NhyI0RMzMzy8qNETMzM8vKjREzMzPLyo0RMzMzy8qNETMzM8vKjREzMzPLyo0RMzMzy+q/uQ7qp9DOfCkAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
## Linear Regression
%% Cell type:markdown id: tags:
The next step is to do some simple modelling, to test the feasability of models. Here we will do this with linear regression. To perform regression modellings we now create a matrix of independent variables (features) given by `X`, and the dependent or target variable `Y`.
%% Cell type:code id: tags:
``` python
X = df.drop(['Y'],axis=1) # drop Y from our dataframe
Y = df['Y'] # create a dataframe with just the Y values
```
%% Cell type:markdown id: tags:
Now we split the data into training and testing sets, and model the data using the `sklearn` Linear Regression model. Recall the method `fit` trains the data, then `predict` uses the independent test variables to predict the target values. This can be compared against the dependent test variables to calculate $R^2$, the square of the correlation coefficient. The method `score` calculates $R^2$ for the training set. This will typically be greater than the $R^2$ for the testing set. As is apparent the fit is not fantastic, but we will see if it can be improved using regularization.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# split into a training set with 80% of the data, and a testing set as the remainder
X_train, X_test, Y_train, Y_test = train_test_split(X,Y,test_size=0.8)
linear = LinearRegression() # instantatiate the linear regression model
linear.fit(X_train,Y_train) # fit the data to the model
training_score = linear.score(X_train,Y_train) # calculate rsq for the training set
# use the independent variables for the testing set to predict the target variable
preds_linear = linear.predict(X_test)
# calculate the correlation of the predicted and actual target variables
rsquared_linear = r2_score(Y_test,preds_linear)
# print the training and testing scores
print("Training score is",round(training_score,4))
print("Testing score is",round(rsquared_linear,4))
```
%%%% Output: stream
Training score is 0.4681
Testing score is 0.4711
%% Cell type:markdown id: tags:
We can also evaluate the model graphically by plotting the predicted test values against the actual test values, and plotting the line of best fit for this data. This is compared against the line $x=y$, which is shown in black, and will alway have a greater slope that the line of best fit.
%% Cell type:code id: tags:
``` python
plt.scatter(preds_linear,Y_test) # scatter plot of predicted values against actual values
# use numpy polyfit and poly1d to create a function for the line of best fit, then plot this function in blue
p1 = np.poly1d(np.polyfit(preds_linear, Y_test, 1))
plt.plot(Y_test,p1(Y_test),'b--')
plt.title("Linear Regression, Rsq=%f" % rsquared_linear) # add a title including the correlation coefficient
plt.xlabel("Predicted") # xlabel
plt.ylabel("Actual") # ylabel
plt.plot(Y_test,Y_test,'k--') # add x=y line in black for comparison
plt.show() # display the final plot
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeXgV1dnAf2/CJSRsYReCCCL7vihatQpWQXGJgCJqlbaWWsUFLR9YN6hQULSK2mqhCi5UAbGIQgUV1IKiLGERBZWdsCqELdtNcr4/Zu7l5mZm7tw1C+f3PHlyZztzZm5y3nPeVZRSaDQajUYTSFJ5d0Cj0Wg0FQ8tHDQajUZTBi0cNBqNRlMGLRw0Go1GUwYtHDQajUZTBi0cNBqNRlMGLRyqMCJysYhsKe9+VAVEpIWInBCR5PLui0aTCLRwqAKIyA4R+VXwfqXU/5RS7cqjT8GIyDgR8ZoDbI6IfCEiF5R3v9yilNqllKqllCqO531E5FIRKTHf03ER2SIiv4nnPS360FJElolIrohstvrbsrimvogcEpHlAftuMZ/D95MrIkpEepnH+5r3OSoiO8Lph4gMF5HioPYvDTi+Q0TyAo4tifa9nG5o4aCJOSJSzebQbKVULaAhsAyYm+D7Vxb2mu+pDjAKmC4iiRTybwFZQAPgYeAdEWkU4ponge8CdyilZpkCtZb5PHcB24C15ikngVeB0RH248vA9pVSnwZdf03AsStC9F8ThBYOVRhzFronYHuHiPxJRDaYs7XZIlIj4PjVIrIuYGbfNeDYWBHZas5mvxWR6wOODReRFSLyrIgcBsY59UspVQTMAjIC/9lD3L+niGSZ959r9n1C4HOKyBgR2Q/McNHeGBHJDpidX2buP09EVovIMRE5ICJ/M/e3NGe91cztZiKyQEQOi8iPIvL7gLbHicgcEXndbH+TiPR2/cWdek9KKbUIOAx0NdsW8z0fNL/DDSLS2TzWwOzTMRH5WkSeCJzJu0FE2gI9gceVUnlKqXnARmCwwzUXAJ0x37sDtwOvKzMtg1Lqa6XUGxgCI+p+aGKLFg6nHzcCA4BWGAPOcDAGX4xZ3B8wZmr/BBaISIp53VbgYqAuMB54U0SaBrTbB+OfvDEw0akDIlIduA34GTgS6v7m+f8BZgL1MWaU1wc1e4Z57CxgRIj22gEjgXOVUrWB/sAOs52pwFSlVB2gNTDH5jHeAvYAzYAhwF99AsbkWuBtIB1YALzo9E6sEJEkEbkWY6X1o7n7CuCXQFuz7aEY7xHg70A+0BT4rfkT2N4GU1Ba/fzDPK0TsE0pdTzg0vXmfqs+Jpv3HQnY5uIRkbPMfr/u7uld9aOHiPwkIt+LyKMWK8ZZYqi6lohIN5f31Zho4XD68bxSaq9S6jDwPtDd3P974J9Kqa+UUsVKqdeAAuB8AKXUXPO6EqXUbOAH4LyAdvcqpV5QShUppfJs7n2jiOQAeeb9hpiriFD3Px+oZvbdq5R6F/g6qO0SjFlmgXl/p/aKgRSgo4h4lFI7lFJbzXa8wDki0lApdUIptTL4IUTkTOAiYIxSKl8ptQ74F/DrgNOWK6UWmTaKN4BwBqdmAe/pP8ADSqmsgP7VBtoDopT6Tim1zxykBwOPKaVOKqW+AV4LbFQp1VUplW7zc5d5Wi3gaFB/jpr3tOJe4Cul1JoQz3Qb8D+l1PbQj++qH59jrFYaYzz3MEqrp24BWmJMFpYBi0Uk3eW9NWjhcDqyP+BzLsY/IRj/RA8GziaBMzFmxojIbQEqmhyMf8yGAW3tdnHvOUqpdKAJ8A3QK+CY0/2bAdk+dYTN/Q4ppfLdtKeU+hG4H0P9dVBE3haRZuZ1v8OYlW8WkVUicrXFczQDDgfNancCGQHbwe+5hsXM1o695nuqAzwP9PMdUEotxViF/B04ICLTRKQO0AhDgAa+l50u7xfICfO+gdQBjgefaL6zezHsAaG4jSBhFU0/lFLblFLbzcnKRuAvGCs4zOMrTHVUrlJqEpCDsfLVuEQLB42P3cDEoNlkmlLqLVMlMB1DddDAHLi+ASTgetfpfZVSP2Goe8YFqKZs7w/sw7BPBN7vzOBm3T6P2Yd/K6UuwhAiCsOgilLqB6XUMIwZ6ZMYRtCaQW3vBeqLSOBsugWQ7fYduEEpVQCMAbqISGbA/ueVUr0wVCxtMWbMh4AiSr+XFoHtmbaPEzY/L5unbQLODnq2bub+YM7DUGF9a9p6pgLnich+CXD5FZELMQTqO2E8fjj9AOM7FJtjbo5rgtDCoergEZEaAT/heuxMB+4UkT6m0bOmiAw0/zlrYvxzHQIQw7WyczSdVUptBhYD/+fi/l9iqIJGikg1EbmO0iqtsJ5HRNqJSD/TnpKPob4pNp/tVhFppJQqwZht4jsW0PfdwBfAJPNdd8VYccxy8+wiMlNEZro5VylVCDwDPGZee675TB4Mb598oNhUX72LIXDTRKQjhgE4sK1OQd49gT93mud8D6wDHjef7XoM29Q8i+79F0N10938eQzDu6i7Ku3yezswL2il5bOp1AA8xqbUMO1LIfshIleKSBPzc3vgUeA9c7uFiFwoItXNa0djrHJXuHnnGgMtHKoOizAGOd/PuHAuVkqtxtDTv4hhJP4R01itlPoWY4D6EjgAdCE2/2hTMIzHjUPcvxAYhDEA5wC3Ah9g2BDCfh4Me8Nk4CcM9U9j4M/msQHAJhE5gTETvilIXeVjGMbAuBfDLvC4Uuojl899JuG9v1eBFiJyDYZqZbr5TDsxjNFPm+eNxFAT7scw3ofyHrLjJqC3eY/JGLYh38TgFhHZBMbKRim13/eDYRPwmp8xz6+B4QRhpVL6Jcbf6iKMVU4eEBiPYNsP4DJgg4icNK9/F/ireaw28JJ5XTbGd3qlUupnNK4RpYv9aCohIvIV8LJSKtIBsFwwZ8brga5KKW+c7zUcuMNUn2k0YaFXDppKgYhcIiJnmGql2zFUDB+Wd7/CRSlVqJTqEG/BoNFES2WPJNWcPrTDiDmohRFzMUQpta98u6TRVF20Wkmj0Wg0ZdBqJY1Go9GUoVKrlRo2bKhatmxZ3t3QaDSaSsWaNWt+Uko5JlOMm3AwXdg+x3AbrAa8o5R63PTvvoRTofHDlVLrzACnqcBVGBGlw5VSa8u2fIqWLVuyevXqeD2CRqPRVElEJGT0fDxXDgVAP6XUCTNgZ7mI/Nc8NlopFRwteSXQxvzpg+Gn3CeO/dNoNBqNDXGzOSiDE+amx/xxsn5fh5nO10x2li6ls35qNBqNJkHE1SAtIskisg44CHyklPrKPDRRjPTBz8qplNAZlE4atofSicw0Go1GkyDiKhzMVMndgeYYCbk6Aw9hpBs+FyP//hjzdKukWGVWGiIyQoxiLKsPHTpkcYlGo9FooiUhrqxKqRzgU2CAUmqfqToqwMj94kugtofSGSWbY+StCW5rmlKqt1Kqd6NGoSoXajQajSYS4iYcRKSRmMU1RCQV+BVGjvym5j4BMjFSP4NRLes2M4Pm+cBRHQGr0VR+5mdlc+HkpbQau5ALJy9lflZMM5tr4kQ8vZWaAq+Zed2TMAq9fCAiS8WoGywYKXnvNM9fhOHG+iOGK+tv4tg3jUaTAOZnZfPQuxvJ8xoZvLNz8njo3Y0AZPbQJsWKTNyEg1JqA9DDYn8/i9Mxq3zdHa/+aDSaxDNl8Ra/YPCR5y1myuItWjhEiNfr5W9/+xu33norGRnxe4eVOkJao9FUbPbmWJcTz87Jo9XYhTRLT2V0/3ZaULgkOzubzp07k5OTQ506dfjjH/8Yt3vp3EoajSZuNEtPtT2mOKVm0naI0EyZMoVu3bpRWFjIyJEjufPOO0NfFAV65aDRaOLG6P7tStkcrNBqJmfy8vK47rrr+Ogjo9Dg5s2badeuXdzvq4WDRqOxZX5WNlMWb2FvTl5EKiDfub427FIk2KmfTneWL1/OXXfdxcaNG2nbti2rVq2iTp06Cbm3Fg4ajcaSWHkaZfbI8J9/4eSlZFsIAif10+mIUooLLriAr74ykkosWrSIK6+8MqF90DYHjUZjiZOnUaSM7t+OVE9yqX2pnmRG94+/mqSycPjwYYYMGeIXDFlZWQkXDKCFg0ajscFO1RONCiizRwaTBnUhIz0VATLSU5k0qIu2N5hMmjSJBg0asGDBAp566im8Xi/du3cvl75otZJGo7GkWXpqXFRAgWomjUFRURETJkxg/PjxAKxYsYLzzjsvxFXxRa8cNBqNJVoFlBiWL1+Ox+Nh/Pjx3Hrrrezdu7fcBQNo4aDRaGzQKqD48/zzz3PxxRcDMGPGDN544w2aNq0YZWy0Wkmj0djipAKK1s31dObkyZOMGjWK6dOnA7B48WKuuOKKcu5VabRw0Gg0YaMT6kXOZ599xqWXXgrA2LFj+ctf/oLH4ynfTlmg1UoajSZs7Nxcx7+/qZx6VPFRSvHCCy/Qv39/AMaNG8ekSZMqpGAAvXLQaDQRYOfOeiTXy/ysbL16CGLz5s106NABgIEDBzJjxgwqerEyvXLQaDRh4+TOGk2QXFXkk08+8QuGCy64gPfffz8qwfDEEyACc+bEqofWaOGg0WjCxsmdVedJMsjNzeXGG2/k8ssv5+yzz2bWrFl88cUXGEUww+enn+CWW+Cxx4ztzp1j2FkLtHDQaOJIVS2Rmdkjg/RUa125zpME27Zto2bNmsydO5dhw4axYcMGbr755ojaUgoeeQQaNYJ//xvGj4eCAujYMcadDkLbHDSaOFGZPXrcuKmOu7ZTmXTcdkFyge3VTfUgAjm53irpAvviiy/y5z//mRo1ajB06FBmzpwZcVv79sHIkfDuu8b2okWQqDRLWjhoNHGispbIdCvUgtNx2w30we3l5Hn9x+IhMMsr/uKnn37y2xI6d+7MBx98wFlnnRVRW0rB5ZfDJ58Y208+CQ88ANUSOGJr4aDRxIl4JK5LBOEINTd5kqzac9N2IG4H/PJara1Zs4ahQ4f6t1evXk1KSkpEbe3eDXfccUowfPwxXHZZLHoZHtrmoNHECTvde0XXycdaqLm5zukc34CfbRYLciotGo80404UFxfTokULevfuTWFhIcuWLUMpFZFgKCmB88+HFi1g+XJ4+GEoLCwfwQBaOGg0caOyJq6LtVBzc50CW4N9OAN+Ildr+/fvZ8CAAezevRuAdevW+SOfw2XbNvjVr8As4cDGjTBhApRnfJwWDhpNnKisietiLdSs2rPCbkUQzoCfqNXaqFGjaNq0KcuXL+fpp5+mpKSE+vXrh92O1wtt2kDr1rB6NTz1FBQXw9lnx7S7ERE3m4OI1AA+B1LM+7yjlHpcRFoBbwP1gbXAr5VShSKSArwO9AJ+BoYqpXbEq38aTSKojLUL3BqaI23P5610JNdb5lwr+0M4dSVG92/n2oMqEgoKCmjZsiX79+8HDNtCp06dImpr82YwY+MA2LQJzjwzFr2MDfE0SBcA/ZRSJ0TEAywXkf8CDwDPKqXeFpGXgd8BL5m/jyilzhGRm4AngaF2jWs0mvgRa6Fm1V6rsQtRFucGrwjCGfBjLdgC2bJlC8OGDfMLhsOHD1OvXr2w28nPh0cfhRdeMLYvvxwWLzainisScRMOSikFnDA3PeaPAvoBvmiQ14BxGMLhOvMzwDvAiyIiZjsajSYCQnn5lGfabbcrgnAH/FgLNqUUt99+O2+88QY1a9Zk/vz5XHfddRG1tXEjdO1qfL7+evjHP+CMM2LW1ZgSV1dWEUkG1gDnAH8HtgI5Sqki85Q9gO9bzAB2AyilikTkKNAA+CmozRHACIAWLVrEs/saTaUmlFtneQfphbsiKA/13NGjR7nzzjt5++23AaN8Z7du3cJu58QJI05h5kxIToYbboC33opxZ2NMXA3SSqlipVR3oDlwHtDB6jTzt9WiqsyqQSk1TSnVWynVu6JnNdRoEoVVmo5QXj6JdvsMpqIb7J977jnS09N5++23mThxIkVFRREJhjlzoHZtmD4dhgyB/fsrvmCABAXBKaVyRORT4HwgXUSqmauH5sBe87Q9wJnAHhGpBtQFDieifxpNZcZuBWAXeObT6VeEIL2KaLAvLi5m8uTJPPLIIwC8+eab3HLLLWG3k58P48YZ0c0AI0bAP/8Zw47GmbitHESkkYikm59TgV8B3wHLgCHmabcD75mfF5jbmMeXanuDRhMauxVAso2F06fTr6xBevFk1apV9OnTh0ceeYQ+ffqwc+fOiATDG29AaqohGH73OzhypHIJBoivWqkpsExENgCrgI+UUh8AY4AHRORHDJvCK+b5rwANzP0PAGPj2DeNpspgN9MvVqpMfIFgrCwunLyUvu0bVcogvXjx/vvvc95557FmzRpeffVVvvzyy7DtmidPwn33wW23GdsvvAD/+hekp8ehw3Emnt5KG4AeFvu3YdgfgvfnAzfEqz8aTVXFzusnw/TqmbJ4C9k5eQinjHjZOXnMW5PN4F4ZLNt8qFy8lSoKOTk5PPDAA8yYMYOUlBTeeecdrr766rDbeeQRmDjR+DxyJEyaBLVqxbizCUQn3tNoKjlOXj8+nf6Fk5eWESB53mKWbT7EirH9Et3lCsP8+fO5/vrrAXjggQf461//GnZepKNHYfRow+AMhvpoxIhY9zTxaOGg0VRy3MQBVATjc0VCKcW0adO48847Abjvvvt45plnwm7n3nsNtVFBAdx4oxG30KBBrHtbPmjhoNHEkUQFmYXy+nETcFaeAXGJZOvWrZxzzjkAXHHFFcyYMYNmzZqF1cbhw3D//YbhGWDlSujTJ9Y9LV+0cNBo4kQigszcRkAH2xygtPG5vAPiEiWYPv/8cy655BIAbrrpJmbNmkVSUnh+ObfdZgiFatXgD3+AKVOMOIaqhhYOGk2ciHcluHAjoBX4BURG0AAcaV9jMagnQjAVFBTQuXNn/6ph+vTpYafXPngQmjQ5tb16NUQQE1dp0MJBowlBpANgvPX8oQZ0q+M+wRBshI6kr7Ea1OMtRHfu3EnLli0BaNq0KWvXrqV2GFN9pWDaNKP4DkDdukZt59QqHg6i6zloNA6EU4UsmHgHmYUa0ONdByFW6TfiKUTnzp1L9+7dAbjqqqvYu3dvWIIhOxuSkuDOO42aC5s2QU5O1RcMoIWDRuNINANgvCvBhRrQ7Y4niZTKwRRJX+dnZVsauMEQoG6EZ3B/3e636ktwXqmTJ08iItx44420bduWH3/8kYULF7ruU0mJUXjHV6qhVSujdGfHjq6bqPRo4aDROBDNrDbeieVCDeh2FdiKlSqzCsrskcHgXhn+lBvJIvRsUZcpi7eUESS+1ZQTbldXAH3bWyfQtNsfiNXKbtRL79Guc3f/OcuXL6d169au+gKG51FyMowZA126wA8/GGU8y7NkZ3mgbQ4ajQPhVCGzIp6J5ULFNwQfTxKhOChdWeAqaN6abP/xYqVYsfVU3stAe4LVaiqYcGwGyzYfCmt/IIF9UaqEvdP/QNGRfXhqN+Djjz/msssuC9mGj5ISI07hnnuM7a5d4dNPDUFxOqKFg0bjQLzLTkZLKOETeLzVWGu1yt6cvLAGfLe2gGjPc3O975zi3KPsnzWGoiP7ADjj9qkhBUOgo0GtI2fwzbReAFxxBbz8sqFKOp3RwkGjcSCeZScTjdMqyO1AbmdnsCI9zZ0eJprVWbP0VDYvncvRL+dQnHeMWl2voP6AkTSvV9PxOp86KregmOOrW7FjmVFqZsANJ1g0u1aFK9lZHmjhoNGEoCLWHIgEp1WQL1AulrhNuB/p6szr9ZLyxcsc/mguAE1/8zzVG5/t6topi7fw04bG/LSgp3G/Nvupf/k3HDszCZHTN9dUIFo4aDTlSCJTVvjaHbdgEzl5XgBqeAyfFKsBOlqOmvdw269w3sOqVasYOXIkX3/9NY3OyKDl757nUFGKq2u9Xvh2YQY5yw0BUvvcbdTr+x0isDcnjAes4mjhoNGUE1ZBZKPnrmf8+5vIyfXGTVgUFJX4Px/J9fLQuxuZNKgLkwZ1KTVA923fyJ/OO5KqW+G4ooYjGDIzM3nvPaNG2Ny5cxkyZIjtucG8+io8/jjk7GlH9TNyaDBwPdUbngi7z6cDWjhoNFES6ezfygjsLVEcyTVm3G4ijsO9t1Pcxoqx/WyvtUr57YRbo304UdbHjx9n5MiRfsHw+eefc/HFF7vqT0EBPPHEqXoLY57+mfdyVlVYR4OKgI5z0GiiIJoIajdGYKeAu0juHalnkF3MhB1u4zncBhlOmzaNOnXq8Oabb/LYY49RUFDgWjBMnWrkRJo40fBE2roVJj/YIK4xKFUBvXLQaKIgmrxAdl46wdgN3Hb3fnDOesB6tRGpZ5CVXSC3sMi/ygkkIz3V9SBr92y+UqYPXt6GbctmM3r0aACWLFniOnYhNxfOOgt++snY/vBDyGucza/nVH7Ps0SghYNGEwWRzsTnZ2VzsqDI1T3sBm6n2tF2qhm3nkF26qrgdODRxoA4Ccjt329i8EOGIBg0aBDPPvus65rOH39s5EPyCYY9e2DVwfJNS17Z0GoljSYKIskL5BtUc4K8edI8SXiSSzvYOw22TvewU0e5SenhVl0Vi/Qgo/u3wyqk4OS3n7Fvxr0AtLzqTt555x1XguH4cahTBy6/3Ih4XrbMcKnNyIhdosDTBb1y0FR6yrOCWSQ++nbRyPVqpvhjDtw8Syj3U7uVRai4jXBUZdHGgGT2yOD+2ev826qokCOfvcbx1YbRueF1Y6H9RYiLqLT33jPKdh4/bmyvWQP16p06rkulhocWDppKTXlVMAsUSOlpHlKqJXE0z537qZOePZzB1nfeg3PWl8mZBJG7ZSZqEPW9Qx+FB7f5Vwu1ew4k/dLfkuRJCfkcu3YZtgWAc86BL76ACy4oe160ebJON+KmVhKRM0VkmYh8JyKbROQ+c/84EckWkXXmz1UB1zwkIj+KyBYR6R+vvmmqDuWhKghWuxzJ9VJQVMKzQ7s7uoP6sBuMxGw7HDJ7ZPDMjd1imho83nUoIOgdKsWJDR+x/03D6Fyrx0DqX/5HkjwpIZ/jvfegc+dT22vWWAsGiH8K9apGPFcORcCDSqm1IlIbWCMiH5nHnlVKPR14soh0BG4COgHNgI9FpK1SKnYhm5oqR3moCsJRu1ipvEb3b8eo2evKBJYps+1wVzyRRBhb9cvXRqh607HA9w6Ljh0k+6XfApDSoisNr36AlDqNKFaqTCnTQLZuNVYJYGRPfekl+MUvnO9ZlfJkJYK4CQel1D5gn/n5uIh8Bzh9C9cBbyulCoDtIvIjcB7wZbz6qKn8lIeqwK1AslN5TRrUxTbiOFKhFo46yi4yGwFvsdEzp3rTsWBvTh4F2d/5VwvJdRrTZOgTJCUls3XSVbbXKQVz58Lvf29sX3IJLFkC1au7u29VyZOVCBJicxCRlkAP4CvgQmCkiNwGrMZYXRzBEBwrAy7bg4UwEZERwAjAtVubpupSHim13QokpxVGhk0bdVNjV1HGzlBvF5kdTHC96VgZ/r1eLyeXTOXQuk9IrtOIuhcMpXb3AYCzUM/Kgp5GnjzOPdeo69y9u+3pmiiJuyuriNQC5gH3K6WOAS8BrYHuGCuLZ3ynWlxe5i9WKTVNKdVbKdW7UaPQlaI0VZt4V1uzwq3u2snwPLp/OzxJZf/kTxYWhW13sMLJHTWc1Ynv3GgiwQPJzs6mevXqHMr6iLSW3Wj22xf9gsFOqCsFM2eeEgxXXGEYnbVgiC9xXTmIiAdDMMxSSr0LoJQ6EHB8OvCBubkHODPg8ubA3nj2T1M1SLSqwK3u2m6F4RMJtWpUKxNh7C1WEdkdgrFbtdw/ex3JFhXh7PDVZIgmEtzHjBkz+NOf/kSNGjUYMGAAtz36Ak8v+d7xHa5YAfffD6tXQ+PG8MorcPXVrm4HlF3tBCYT1DYHZ+ImHMRwTH4F+E4p9beA/U1NewTA9cA35ucFwL9F5G8YBuk2wNfx6p9GEw1uBJKT4TnQtz+YWBjTndqwEgyeJKEEKA5SL53IL3Jcbbjp64kTJ6hduzYAzZs358svv6Rt27YAXN+zueU1JSUwfboR5Qzwwgtw112QFIauw8q28ubKXf7jOkLamXiqlS4Efg30C3JbfUpENorIBqAvMApAKbUJmAN8C3wI3K09lTSVmcweGXFNdR1tG8kiflXclBu6UTul7FzRW2KsZCJ1b920aRMXBPiWfvvtt37BYMdHH0GHDoZgaNMGli6FkSPDEwwQXq1rTVniJhyUUsuVUqKU6qqU6m7+LFJK/Vop1cXcf23AKgKl1ESlVGulVDul1H/j1TeNJlFkhDnQx8qY7iaLaolSbJ880B+bYVecZ69pIwknRkApRc+ePencuTMHDx5k4cKFKKX8KwgriouNtBdXXAHff2+sHLZsgb59QzysDbGudX26oXMraTRxxG2q61gb0wMN9XYEz/qdVgfB7SWL+GfdwUbpw4cPM2jQILKysgDYsGEDV11l754K8N13cPHFRsI8gJUr4Y47iKqWs9sVmI6QtkYLB40mjrgZpDPSU9k+eaA/r1KrsQu5cPLSUoPu/KxsLpy81PKY071XjO3Hc0O7u5r1h1odZPbI8J/js1sEey1NmDCBBg0asHDhQsaPH09RURFNmjSx7WNRkSEUOnY0BMSbbxr2hj59Qj5eSNwIZh0hbY8ot1XAKyC9e/dWq1evLu9uaDSusEtxPWlQF4CwjnmShFo1qtmWE43USyfUdXY1HJrVro53/p/x/T+uXr2aXr16Wbbpu3dWlhHMtmaN0caPP0Lr1pG+XWvmZ2Uz/v1N/j6nepKo4UmOaxnWyoCIrFFK9XY8RwsHjSZx2A2UdmU4fSuOUEWBfIIks0eGoxAKZyC0aseKoqMH+On9ZyjI/haAvXv30rRpU9s2UlR1vn/qcgDq1zeC2QYPdt2tsIjVu6hquBEOOiurRpNA7Fxgo80RFeh1Y5WlNdyYBHDn7XNk2asc+/pdAGbNmsXNN9/s2EbBvrpkv9fTv7127amMqvEgFvEZpytaOGg0FYBQKTnclBP16f/tAtzC9cpxOr+kMJ8jS6dzYv1iACZMf4ebby47/fe1Uaxs5YgAACAASURBVJJfjd1TjUTLybXyaHD1Ov71l0acdVZ8B2hdwyFytEFao6kAOBmD3Xo8+TyI7AjXK8fu/JItn7L72SGcWL+YjEuGMfer7Tx8h7VeqFl6Kvl76rF35sUAJNUopNkdn1OrU3ZC4gsSkX68qqJXDhpNBcBNSg7fsbqpHk4WFvkzqIIhSJwEQyReOX3bN2LWyl3+QD6lFPnrPuDgkn8C8Nxzz3HffffZXn/gAHzxkJG0L7luLo0GrSKtzUH/8UTM3ssjMWNVQQsHjaaC4JSSI/iYlWHbV4shmGSRiIzR89Zk+wVD4U+7+HnRcxTu+56ePXvy5ptv0qFDB9vrP/kEfvUr43O1WgU0/e3nJFUvLbwSMXvXNRwiRwsHjaYSYidIgmfJgpFLyafCcTsoBhpy83as4+DsRwBodc09rH5vKiJiKaB6Ncxg1CiYN89oZ9IkaN//Jx56FwIDsBM5e9c1HCJD2xw0mipESrXS/9K+mb/bFNu+YLvsnDxKvPn8vOQfHJz9KFKtOg2ufhDVsb9fMASn8L7nyf20aGEIhma/3EmLB/7L+ywFKJVWvZ5Zc3vU7HWuA/o0iUevHDRVglgVokk0seq3m5iEUC6cgW3k79rAgbf+DECtbgOod9kdJHlq+FVBgSsL7+E0Dn/UmfwdjUiqXkSjflvwdNsBlK5+t2JsP9vqeKAzo1Y0tHDQVHrKc8CJZnCPZb/dxCSAsxHY18aJTcv4+QOjBldah1/SYMBIoLQqyNfO4aUdOL7qbADq/uJ76l6wFalWUqrdQKHkVGdiyuItlUaonw5o4aCp9JRXoFO0g3ss++3W88fJCLxrTzZ7/n4bACnNO9Lw6gepVtfIixRcR7phch2+nXcOuVuMSOgGA9dRq7O9esjXP6d+Br+/yroarCpo4aCp9MQ60Mk3KGXn5PmrpgUPjhD94B7LftsF0QXiZARevXo1+/5lVNZJad6RJsMmIUlGbEVgHWml4O67YfVLF0NSCemXbKbOeduQJEWqJ5kaniTr3EumUArVz8BIb61+Kl+0QVpT6YlloFOgoRWwzT4K0Q/user3/KxscguLyuz3JAvpqR7HdODFxcV069aNPn36UK9eOs1v+gtn3PKUXzAECpTsbLj2WnjpJePaidN/puOAvSQlKX/7j1/TyTGzq5uAvr05eY6CV5MY9MpBU+mJZaCTk+4+eFUQKuWFDzv1SCz6bWeITk/1MO7aTo6z7P379/sT5AFs2bSRz3fmlenrtd0yuOACo8aCxwPPPgv33APJyY34M/0s27ZTBwXGHditIJqlp+q0FxUALRw0lZ5YBjqFGnwCj7sZ3N3YJaLpt50wq5lSzbGdRYsWMXz4cAB69erFqlWrEBEy65dW2/zzg/2kX5/D8Z3pADwx/SD3397YsU+h4gp8x+0ypjoF9Om0F4lDCwdNlSBWgU6hdOKBg5PblBdOdolo+x3uDLugoIAaNWoA0KVLFz799FM6duxY5ryiImjXpZBtW5ogniLqXvADdS/6npk/JtMuK3bV6sD+/em0F+WLFg4aTQBWqwEfVoNT4Cx4yuItjApyyYy3esROmKWnecrs27JlCzfddJN/e+XKlaSlpZU57/vv4bbbYNvm6sY9fvc51erkA7H3ArMTjjrtRfmjhYNGE0CwTtzJW8mHk+rIrV0iUkb3b8fod9aXSsIHcCK/iPlZ2WT2yEApxcUXX8yKFSto0KABCxYs4JprrinTVmEhtG0LO3dCejrUu/Q7ap+3rUwd50Tp/XXai/JFV4LTaKLEroqbzyjsVInMjS9/qHO6j19CTl5Z99GM9FQW3tmTzMxMPvvsMwC2b99Oy5Yty5y7aRN07nxqe+9eGPKa9XMli/DMjd30wF2JKddKcCJyJvA6cAZQAkxTSk0VkfrAbKAlsAO4USl1REQEmApcBeQCw5VSa+PVP030zM/KZtyCTf6BqV6ah8evcfaQidV9y1vdENgHu+mV771MGtTFsr9ujNWPzN9YKm124DlgrHCsBAPAls/fo/vLv2XPnj3cfPPNzJw5E4+ntLopNxcefxymTjW2r7/eyI0kYq9iK1ZKxxycBsRt5SAiTYGmSqm1IlIbWANkAsOBw0qpySIyFqinlBojIlcB92AIhz7AVKVUH6d76JVD+TE/K5vRc9fjLSn99+NJFoaee6arYvaR3re8awK7ra0MpQPIgnGqG+3LQzRq9jpL4ZOe6qGgqMSyD6qkmJzPXvOX7/zyyy85//zzy5y3di306mV8HjbMEBCNGpU+Z35WtmXZ0VDPpqnYRLVyEJH3wXZShFLqWqeGlVL7gH3m5+Mi8h2QAVwHXGqe9hrwKTDG3P+6MqTVShFJF5GmZjuaCsaUxVvKCAYAb7GynenGYvCuCDWB3eYxAnv9/PysbFuvqOycPFqNXUiSSMhVSTCFB7dx+JPpFOzaSL2GjVm/djVnnnlmqXOOHoX/+z945RVje/hwmDHDXA29UnaFM2r2urCeTVM1cFIrPR2rm4hIS6AH8BXQxDfgK6X2iYjPaToD2B1w2R5zXynhICIjgBEALVq0iFUXNWHiNDAED2ixHLwrQnBUOPeyMjz7Vh5OKLCtBW3Hzx++yIn1HwJwz/hnmfrofUiQNfmVV+COO4zPw4fD3/4G9eqVr1FdUzGxFQ5Kqc9icQMRqQXMA+5XSh0L/mMNPNWqGxb9mgZMA0OtFIs+asLHTS6fQOLtupnIgcrtswtY+uWHs/Kwazc9zePPYVTiLeDIslf9guGDDz5g4MCBpa7JzYVHHjGimwFGjTIEg1OffEI9lhHoFcFepHFHyNxKItJGRN4RkW9FZJvvx03jIuLBEAyzlFLvmrsPmPYIn13CV1R2DxC4/m0O7HX7IJrEMrp/OzxJtoK+DLF03XTK3ZMI3OQHAmNmYzXwRSMoBbjl/Bb+HEYnv/sfu/82mBNZC6l//iBmf/ljGcHw4otQs6YhGP74R0OtFCgYnPq0NyePzB4ZpYr12OVpCoVVgSA3BYg05YMbb6UZwOPAs0Bf4DdYz/JLYXofvQJ8p5QK/FNcANwOTDZ/vxewf6SIvI1hkD6q7Q0VF9/AEOytNLBrU+atyY5bZGu8gqPCmdEG9yHJjIUIJsMhsV44q65kEUqUKtUvpRQfzn2dfy54EoB2t4xj8oN3lOrzsWMwZgy8/LKx/cYbcOut1vcItSKLRcxBRbAXadwT0lvJtGr3EpGNSqku5r7/KaUuDnHdRcD/gI0YrqwAf8awO8wBWgC7gBuUUodNYfIiMADDlfU3SilHVyTtrVQxqWyqg2g9oOyuH9wrw9JrKxxvJ0+SMOWG0jEFP/74I23btkUpRf/+/fn73/9O69atS133wAOlVUhPPGGsHuL1DtzQauxCSwO7ANsnD7Q4ookXsYpzyBeRJOAHERkJZAPOmbcApdRy7FcYl1mcr4C7XfRHU8GpbJGt0c5orVYzfds3KrWCsvLaGv/+JsvaB4HUqlE6gd6SJUvo378/AA899BATJkwgKemUdvjIEbj3XnjzTWP77bdh6NCQj5CQdBUVwV6kcY+blcO5wHdAOvAEUBd4Sim1Mv7dc0avHDSxwG5GC4ZqKJLBMlQMg4/AwkJW+GbVXq+XJ554gokTJ1JSUsI///lPRowYUercP/wBZs+GEydg0CCYPh3q1nXV3ZDEYjVYEWJUNAYxWTkopVaZH09g2Bs0miqF3YxWwL8/3HiNWLncNktPZdu2bX610fDhw3nhhReoVauW/5xDh+Cuu+Cdd4ztNWugZ8+wbuNIrGpd62R6lYuQwkFElmHtUqpDIzUVHjczXitXTSG6eA03KpRQtodUTzK/SPqBnj0HATB48GBmzJjhP64U3HgjvGe6dIwYYdgZLBKtRkUsDcmVTeV4OuPG5vCngM81gMFA2ZqEmipFJGqEimaIdjvjtZrROkUv+7KdBt4n+LndxAY4xTs0rlbAqgmD2Qz06dOHf//735x99tn+4/v2QbNmp87/5hvo1MnliwmTihB4qEk8IeMclFJrAn5WKKUewHA11VRRIvFHr4g+7OHUIc7skcGKsf3YPnkgK8b2s3VDBUo9l91zAyFjA+wG18IDW1k1cTAAkpTMn55/2y8YlDJqOPvq8zRpAvn58RMMYG8wThLRMQpVGDdqpfoBm0lAL4xMq5oqSiRqhHCuSdQKw2nGG6oPTkV/Ap/L6blXjO3n+FzBKxSlSji8+B+c+OZjkmvVp855g6lz7nU8+v5mqnk89KifgS/b9kUXGakw2raN//vU2VlPT9yoldZgqF8FQ520HfhdPDulKV8iUSO4vSYa42a4g6BTlbRQffD9vj9E0rloVC6Bg27xyRz2vGhEqHkan02ToU+QnGa4GuUWFHPfQ/kcXmFc16EDfPopJCfHzljshK8dq+ysOoit6hJSrQR0UEqdrZRqpZRqo5S6AlgV8ipNpcVOjeDkj+72mnBUPYFEorayS7WhFK76kNkjwzHK2er5go874UtLkbLzS/bNuAeSq1GjZQ+aDn/OLxjydzZg15SB7FrcmvPPh+3b4dtvDcEAkb/PcMnskUGJjdu7tj1UTdwIhy8s9n0Z645oKg6R5C9ye02kM+1IBkG7nEBHbdJdW/Uh1HNFk+upsLCQ63s25/u3J1K9JJ9mtz1Lk6FPIJKEKoFjq1px4G2jDkNKxhG+77mQS19eSPfxS/xC0e69hZOewy12Ak9hxHVo+0PVwqmewxkYKbNTRaQHp6Kd6wAxdpbTVCQi8Ud3e02kUbKRChUr10m7oLMkEVqNXViq76GeyyraOaVa6DnX1q1bGTZsmH97165dXPevDWTn5JG3rSEH5xo+H6mtD1D/8k1Uq3uqvzl5XkbPXQ84x2gEe1VFi5MdJh7qLE354mRz6I9Rta058AynhMMxjBxJmipMJP7owQOpb1YfytDrZqYdy9QLTgZWKDvQuXkX+d4S/+ecPK/jQHnllVfy4Ycfkp6ezpw5c7jhhhsAGNWvHX8cc5KfPm0LQN1f/EDdi77HKsu9t0QxZfEW+rZvxJsrd5U5riDmtoDA79fqu9D2h6qFUz2H14DXRGSwUmpeAvukqaS4MY5GGiUbqVBxMmI7ZVUNZ6Bz66l1/Phx7r77bj780Ki7sH79en/Bqtdeg+HDjXMbdDlA6i830qJ5Etk59vfNzslj3hp7VU48bAE+QWmXckTbH6oObryVeonIJ0qpHAARqQc8qJR6JL5d01Q23A6SsViVuBEqoYSV79pWYxdaXu92oHOj8nrppZe46667SEpKYsyYMYwfP56UlBQKCoyMqRMnGuc98QQ88kgToAlgn6MJjFTeTpld45nQTifRq/q4EQ5XKqX8aiSl1BERuQrQwkFTinhH0oYrVNwKq2gHOqfrS0pKGDRoEO+ZOS4+/fRTLr7YyHb/0kswZYrhgdS9O8yZA23alG5jdP92jH5nPd7i0vN0T5JY1vD2Ee9KbbGsDqepmLgRDskikqKUKgAQkVQgJb7d0lRGKtps0q2winags7v+dz3rMmDAAD766CPAMEKfffbZ5OXBY4/B02aV9g8+gIEDjYF5+GTrgTnQ4J2e6mHctZ1sdf/JIiEznVoJAcB1zIROolf1cZOy+/+AazEqwoGRmXWBUuqpOPctJDpld8WioqVkdps2G8oOln3bN7Is1GNH8PVtDizjtecmADBt2jTuuOMORITJk43keAcPQr9+MHMmnHmm+3cXeJ/0NA8n8otKrSDcvG+re3mShCKlsBoOrN6XpnLjJmV3SOFgNjQA+BWGx9IRoKlSqtwL82jhUPGIdSqHaNqLVFhFI+QKCgoYM2YMU6dOBeD999/n6quv5sQJqF371HkffwyXBZS8ciPI7LK4pnqSyPeWuH4/TnYMK3SltqpHrCrBAezHKPV5I0b6DO29pLEklimZo00NEanqI9IU1e+99x6ZmZkA3HvvvTz55JPUqFGDxYvhzjtPnXfgADQOqqXoRgVml8U131vCs0O7u37vkdSU0Jx+OAXBtQVuAoYBPwOzMVYafRPUN00lxG6mH8kKIBZ1BCIRVuEa1pVSvPrqq9xxxx0APPzww0yYMIGjRyHVHFfbtoX//c9ImGeFG3uN7f0JL6bBKSV5MNrIfPritHLYDPwPuEYp9SOAiIxKSK80lRK7mf7qnYdD1lO2orzqCLgt1DNl8RZ27dzJ0f/+jeM7v6Fr16688sor9O7dm3nz4L77Tl27di3UrGl/TzdGcadBPZx34hTpHIgbw7am6uIU5z8YQ520TESmi8hlnIqS1mjKYDfTf+ur3RElh4smqV00hMqX5BOCW79Zy56Xf8vxnd/QqO9wBj72GiPe9iICQ4ZAcqqXr782ajA4CQawzwMVHF1u9w9o907mZ2Vz4eSltBq70J//KPhe6akePMmlW071JPPMjd20YDiNcYqQ/g/wHxGpCWQCo4AmIvIS8B+l1JIE9VFTSbCbvQZHH4c630eo2bSTqioaQ3YoW8WTizaRvfifHF+9APGkkP7L20nrfS3/eqOYQ+9dYLaiqDF4GdnVOnEukavAgp/jF63r88XWw6Wik+1UP26DAK3uo91SNa68lfwnG4V/bgCGVoQa0tpbqWJh5wWTbJGeAty5SDrZMOw8ioC4udSuXbuWXr16AZDW/mIaDBhJ0bEm7Hv1EgCqn5FD/V9tIiUjp9QzRlp21eo5BvfKcOVm6+QBNbp/Oy0MTmNi5soa4c1fBa4GDiqlOpv7xgG/Bw6Zp/1ZKbXIPPYQRhGhYuBepdTiUPfQwqFi4TSYBdocfPujGaydBj6wTlkdib/+/Kxsxi3YRE6el9zvv+SnBU+hir3UOKsrjW6cSO53zfl5YVdQSaS120fDa7OQpFP/UwI8O7R7RMIqmjiN0f3bMWr2Osv8R777V5R4FE3iiaUrayTMBF4EXg/a/6xS6unAHSLSEcMzqhPQDPhYRNoqpZwtZpoKhZM6pvdZ9WM6U42kjkG4huz5WdmMnrue/JPH2PO8kV67+hnn0Pja0ZDfmd1TfgFASsZhGlzxDZ7Gx8u00Sw9NWKvq2ir66V6ksgNyBbrQ7AvdqSFg8ZH3ISDUupzEWnp8vTrgLfNFB3bReRH4Dx0UaFKh53raCzjHyA8d8zAa8JhyuItnDywnYNzxwNQrW4TmtwyhZPfnM3hxV0BqNkxm663bKFfx0bMW5NraR8ZFaLUqFN/3dSdsBM+Vqm+AdvVhM6oqgkknisHO0aKyG3AaozsrkcwigqtDDhnj7mvDCIyAhgB+NMda04v5mdlc7KgKKxr3BqyfSilWPPiPRRkf0tSzXQaZf6ZpBrXcPCtdhTsrYekeOlw6zdMvLsxmT0MFY/d6sguB1IoYTW6fztGz11fJsFeYN0Jq6R8p57B3btx2x/N6UWihcNLwBMYk5cnMIoI/RZrF1nLP22l1DRgGhg2h/h0s2pRlTxR7FJI1Evz+BPTWeHTp7uJuv75559p2LCh/9qmw18gd0sPjszvBED9/huo1W03JwUeeveA/1q71ZHbxH5W+Z1COY97ixVJAlYJWu0cAeqlecj3luiMqhpHEioclFIHfJ9FZDrwgbm5Bzgz4NTmwN4Edq3KEm0KioqGXQqJtOrVHIVDoD3ESd/+6aefcuuttwJQp+EZVP/lfzg4uyPen+qQXDeXhlevo0bzI5bXgn0CvzxvsX+wTk/1UFhUzP2z13H/7HXUS/MwsGvTMoGCs1buslUBBVKirA3Mg3tlMHvV7lIrC0+y8Pg1nfzvoipMGDTxIaHCQUSaKqX2mZvXA9+YnxcA/xaRv2EYpNsAXyeyb1WVWKSgqEg4GWntVg/10jwhr88+fIImTZpw8OBB2rRpw1Ov/5dH/+8Cjr1bF4AGA9dRs1O2pR4/OyfP71kknFryZufklSrhWawUnmThWJ6XQDPxkVyvbalPt0wa1MUyBffsr3dbNhprG5Cm6hE34SAibwGXAg1FZA/wOHCpiHTH+BPdAfwBQCm1SUTmAN8CRcDd2lMpNpRXCop44ZTawqowjidZGNi1KRdOXmpbErTo6AGOLXqG4wcPAvDqqysZcFM1CvbXAaDp8P9Rvckx2z4Jp7ykQg3odvaBaEhP9VgO9hdOXlrGXuEtUYxbsEkLBk1I4umtNMxi9ysO508EJsarP6crFa0AT7T2Dyf9vZUrbd/2jUqpa4IFw6H//JXc778grVZtXn/930yaNIyLL4aktAIaXreGtHb7bb1+gFIrhXgQqn1PkjDu2k6Wx+wmADl5Xn8ajUipSnYsjTVOuZU0VYBQeYISic/+kZ2Th+KU/WN+VrbrNtzkIArkg/X7LG0U4i3g0PzJ5H7/BQD/mracp58exnffGcd73reKmu3LVzCkepK55fwWpZ711qDtKTfY5z9ymgCEymvlRCy+R03FJ24R0olAR0i7I56zvHDaDifiN9K+uMk2evLbz8hZPovinH0MHJjJBx/MAlJp2hRefhmuvdZ9W3YeQdGSLBJ14rv5WdncbxNjEU0Bn3h/j5r4U94R0poKQryMj+F6QjlFNftsAtEILztPJh9KKY59NY+cz2YC8PzznzBu3KnBbO1aOOMMSvXfJ/jshv9ipcp4CvlWFBnpqbRskFomUZ4nWaiWJORZRC9D7FJZZPbIKFV7OpBo1Iqh7Fha5VQ10GolTcQ4eUJZYTcg+Qy60aoonIzs3p93c3DuOHI+m0lSal1gM/fe24/atWHWLCNgbOW+0umtAVaM7cf2yQP9OZuC8am1Aj2i6qZ6eG5od1aM7ceOn8sKFm+xon7NFHZMHsiOyQO59fwWJJv6q2QRBveKnTB//JpOMVcrOqVS1yqnqoMWDpqICdcTysr+AWX19m5qPVhhN2idXPEme//1R/K3r+GMX4yhZvWfgHZ06AAbN8LNN4fWo4ey3eQHrAJy8rz+a93MsuetyfarpoqV4s2Vu+j46H9DDqhWtRqCCddG4wandxHuhMEKN8+liT9araSJmHA9oQJVNcExAcFEomoKdmVVxV5yPn+DY1+/a57xJvu/uIVzzoF/vwlXX33q2lDxIE5JBS+cvNT22lDvyE4VlustYfQ76wFrFV04Kr1YqxWd3kWkeaR8VLWgzcqMFg6aiHGbFiIQ30BlZ9T0ERg74HaAyOyR4U+vnbdtDQfnPg5AjbOvJ3/bG0BNunSBlSshLa30tW5WQXaDrNO1dum6fe/IURVWrGyDFe2E2YNz7AVKLLF7F9G6Tle1oM3KjFYraSImGpVFqJlkpKqmo3leTmxa5hcM1Zs8Sf62d4GaTJ0KGzaUFQzgrEd/ZP5GWj+0iJZjF9L6oUU8Mn+j62tDvaNQg2a4qrtipcpVxx+t63RVC9qszOiVgyYqIlVZRJJyO9QAsXfvXnY9M4iSokI8DXrg/fnfFB5oT90LfqDDVXu4996+ttfarYJaNkgtkwLDtz0hs4vjtb4B0ekdhSrK4yR47N5fpDPtWHgZhSqxGoqKFrR5OqNXDppywc447YTTALFy5UoyMjIoKSqkWu2b8f68CqQtDa5cT9PLtjJmYFvHtu1m+Cu3HbE8/62vdpe6dnCvjIg8jjJ7ZHDL+fap5/u2b2S5P9T7C1fwxtLLKLNHht/La8XYfqWSEoYyNFekoM3THb1y0JQLvgHjwTnrLYPIgo3VdgNEcXExzzzzDA8//LC5ZxJFx8fS4oqtSNctZDSowej+7lRdVjN8uyCywD5beRzNW5NN77Pqu7rvhMwuLNywzzIeYdnmQxZXnHp/dv1LdgrttiDeun63huZoVx6a2KGFg6bc8P3DWyXLG3rumSzbfMhxgNi3bx/NmjUDoGnTIezbNw2ox5w5cMMNrYHWYfcpWLVi51EVOPjGYmDNsUk37qRKy+yR4Up4uSHeuv5w3pHOGFsx0MKhklKlolCDxzFlVFXz6fStWLRoEbfffru5dRGHD8/hmWeE++6D5PC0VX6sZrfJSUKxRSWdYX1OlR+JRcRwpLr2DJvr7IL27Ii3rl8bmisfWjhUQhLtCx5PQTRl8RbLtNI+z6Tg+/ZrXZu6dY0aC7VrdwM+BzqwciV07x59X4Jnt8UliprVk8n3llCsFMkiDOtzZinB5TSwuv2uInELjua6eLVjhzY0Vz60cKiEJNIXPNaCKFjQ2BlOffcJvO8D0xax/eUR/nOKilby4os1+OMfIcmla4WToLObxeYWFjsmqevbvpFlsZ6+7Ru5/q4i1bXHSkcfb11/vIWPJvZo4VAJSeQSPZaCyErQOOn0fecppTiy9F+cWLcYpAGoPwFj+eor6GKveXJ1/0BBF+ns1s5o7LOZWBHL7ypWOvp46vq1obnyoYVDJSSRS/RYDm5WgkZh7ZnkO68k/wR7X7mb4hM/A02h+pc8OuYsxo1zv1oAQzBYeUYFCrpQs1u7VYdTtlm7lN7B31W8VYUVwUalDc2VCx3nUAlJpC+4UxBWuNgNor701oHxBcki5P7wFXtfuY/iE0eBi4BdtPj9j/zlL+ELhofe3WjrwePrl1M0s1McgNO7sLqn1XcVi4R1duhMqZpI0CuHSkgil+ix1BXbrXiCi8QUFxcz6LzWqKICACTlf6T/oj61z13sWJkNrGfIoeo8BA7udrNbp8Hb6h0FkyxCiVK235WbFVqks3+dr0gTCaedcKgIy+tYkKgleiwFUd/2jZi1cpdjcNuePXvIzLzVLxhgNxm//47kmtsBexfN+VnZZQrbBBu1rXCjNgLnwTs426wVJUo5GrVDqQqjUTtpN1JNJJxWwkGnA46MWAgiXxRxoGAQKJVm4tZbf8OsWTOBmsAManf6FfWvXu8/327F4lTSM89bbKv3TxYpozaK1FgdKttseprHMQV5qBVaNLN/7UaqiYTTSjjo5XVi8M3AAw2yVgO0wvDoycvLo9/Vd7Fy6UwA6nVczNS/t6J2XcWUxamlBlSgzCAbSm1kVcozK9Kj6gAAG9ZJREFUuBRnqL8Nt+o1q/M8ycKJ/CL/qsZqUhJqhRbN7N/Nik2jCSZuwkFEXgWuBg4qpTqb++oDs4GWwA7gRqXUERERYCpwFZALDFdKrY11n/TyOv4Ez8AD8w1ZseWzj0l76DJz615qdRtBnQE7+OsnG5g0qEspW4Td7N5JMIChivIJEbuZe6i/DTfqNZ9QDFytZKSncrKgiJy80ukx7GId7CYpkc7+3azYNBor4rlymAm8CLwesG8s8IlSarKIjDW3xwBXAm3Mnz7AS+bvmFLVl9fh2FPiZXsJNYv3oZRi/6w3KMyeA0CNs1+j0bVNSErZAVgPnnazezu1EZyaIYdSjbn523Bqw0oo+u4dbXU0iNwxwM592C42Q6PxETfhoJT6XERaBu2+DrjU/Pwa8CmGcLgOeF0ppYCVIpIuIk2VUvti2aeqHKUZjj0lFrYXK9WRXZ6fYLxHT3DwrVcoOvoRALW6fUCDAQBFpc4LHjydCtzYBdPleYu5f/Y6xr+/icev6WQ509+bk0fdVA+eZCmVANDt30aoGIpYTEoidQzQq2VNpCTa5tDEN+ArpfaJSGNzfwawO+C8Pea+MsJBREYAIwBatLDPg29FVY7SDMeeEq3txU51FKouNMCh+d+Qu2UsACnNH6bRkF+QnGJ9RfDg6ZRuIzCYrl6ah6O5XkoCjh/J9ZaqyRz8DDl5XjxJQr00Dzm5Xtd/G4/M31hGnx+Im1KhbonEMaCqr5Y18aOiGKStvNct/9+UUtOAaQC9e/cOLy8xVTdKM5wZYiSzycBZdpKDGscq4hmg+CQcmrOYgoMvAFC79zjqX9bb4kwDt8be4HsnCShFKcHgI7Ams5WA9JYo0qpXI+uxKyzbD2Z+VrajYIBTpUKhfCYlVXm1rIkviRYOB3zqIhFpChw09+8Bzgw4rzmwN8F9q9SEM0MMdzZpt1Kwwxfx7FM57Z17jPytwwDo0uU3HLnkapJrplheK2Y/rAZPd/EElDH+BuITgE4C0q09ZsriLY6CwW2p0HhSlVfLmviS6PQZCwBfEv7bgfcC9t8mBucDR2Ntb6hsuCmpGEg4KTXCTb/h1sjswxfxvPpPA6n30XG/YOjf/w42bHiVFhnpttcFl5YMxleCMtx6BT58AtBOECpg1Ox1rlJNOK20ksRo7f7Z62g5diE9/rKkXNJVVJWgT03iiZtwEJG3gC+BdiKyR0R+B0wGLheRH4DLzW2ARcA24EdgOnBXvPpVGYgkF45TXiArUqqd+urrpXkczw3HeJnqSeZPV7Tj5psP0rTpZaxZcxN16nTm4amzOX7pMFqNXUhuYRGeJClzXTiqjkjUIp5k8V/nVIM5eDVgl+OobqrH/mYK8rynlFs+m0ciBYTOqaSJhnh6Kw2zOXRZ8A7TS+nuePWlshGpwdiN6sIqmjjfa6WhP4WdGio4wC0jPZU7enVgysj9fPllTwBqdb2FVpk3M3s/eEuMNo7kevEkC6meJP8AWsPjbp4yPyubcQs2OaqOfATaPuqleUp5K7lRUQWSnZNHq7ELS82+nfI8hbJ5JAId9KmJhopikNYEEA/3w0DX02BCDRh2Rs3A1YZScN11Jfzhscfxep+CpJrU7ZNJ+i+HkVNQVjPvLVYUBbiNHsn1hnSnnZ+Vzei568tUjnNih0M+I58wbTV2oaPtwIdv9j1q9jrb2s2hiPQ7fGT+Rt76ardtNbpw7qXdWDVu0MKhAhJr90On3EM+QhWyX73zcKnBKTDCdvduaNHie8BQ2dRueT51r7mH5LS6jv2yU9/YCQerkqJOuHlf87OyHb2vrAh1ppgeU5H2KZhH5m8sVWmuWCn/tpOA0G6smmjQ9RwqILGu1+DGoOw0YPhSMASmwpi3Jpt312Tz4ovQrt0S4AIAOnbsRP0bHw4pGOxwElJu1D8+3LwvpzoPITKDO9+7WlIZmwqUtnmEw1tf7Q5rv49E1v3QVD20cKiAhGtcDkUoNUKoAcNKuBw7VJ3BvdO55x4hL68/bdqcwbp169i06Rsy6qVF1E9wdqd1GrCT5FQ6b1+J0SmLtzgaX+2EZrIIzw7tznNDu0ckJPK8JUy5oRvpAQbremkepgzpFtF3aLeqCbXaifXfkeb0QquVKiix9It3iixOT/Uw7tpOjvcKFC6qWDj29dnkfKGA8wHweDysW7eKtDRDKNhlJvUkCbkOxm8nITVuwSZHdU6JKnvfUGlB7IRmiVL+9NthR1lyKvAtVt+fU8rxUFTVoE9N/NErh3Ig3BiGaNvv276RrdtmQZGzpxKcms3nbm3MrqevIufz0VDUkeQaO3n99dcpLCz0C4bgzKRgzFinDOlGPZvAN985drPa+VnZIb2TMtJTwy61GaoEaiSG23iobYb1OTOs/RpNLNDCIcHE2/fcqv15a7IZ3CvDcqbppk7xA5e1I3fVORx6px2GNv4DAF5+9xN+/etfW94bSmcmzeyR4TjYOgVnheqf7x7heueE0smHMtwKcGHr+nFX20zI7MKt57fwf3/JItx6fouQ3koaTTRotVKCibfvuV37gd4uwTgN2vPmwZAhGcDXJFUfQEkhJHlq8Pbyb7nhvFau7n3/7HVMWbyFuqke2xWA0/M79S8wfsHOVddukA+VWsKqSE4gCtjxc16pmhPxYkJmFy0MNAlFC4cEE2/f80jasRo8Cwvhr3+F8eNLgN5AFs0aN+ett/7HRRddFPa9s3Py8CTb68idrrWzmdRL85RKkhdJkjk7nbxVkZxw+63RVGa0WinBhNJzx6t9J04WFJWyf6xeDb17w/jxioyM64AsAJrc+jS//uCopZ3EFy/ghLdYYeHhCRiz8JZjF9Jy7ELa/Hlhqfbt1D+PX9Op1L7MHhml1GfB8Rjh4DaflGMKDY2mEqNXDgkm3imUQ6W1tsKn6snOyWPMnG849K8zkJL/csklb9P+wgtZ9M05SPvL+EmJ/7xALyCneIFgShRl6jkH4y3BH4Hsm9k7BeH5sIvH6H1WfVdpRQLVS25jKkI5DOnEd5rKil45JJh4+54Hth8OBfvroEog35tLUtNbOHz4Gnbuy+KTvJYkdfgVEjQKBhqyw8na6nte3/M74WvfbtAPXr2MW7DJ0ubx4Jz1jp5hVkZ8t/ENObn2XlSnS+K7eHvfacoHvXIoB+Lte+5rf35WdsgcQCWFyeR83o7ja1pS94L3yd36IN6D27n33nv5uuGV7DsZOuWGW727VT3nlmMXhmw/lItqqOR5gZXqrOIe7Oosh6pqB85qvEQnviuPVUosSs5qKiZ65VCFyeyRUSpKNxARyN/ZgH2v/pLja1pRq8c2Tm6+i+LjP9NoyOOsbnqdo2CAUwNjepr1PdJTPVGtkELFG/gGonDSali57tq17ytaJBjPEmxQD6UOTGTiu/JapYQbW6KpPOiVQwUl2lmg7/qcPG+ZGbAnSTj8RWsOf9aO5LrZNL5pNalnHadg/2iSa9WnWq36IetB+wbG+VnZnMgvKnPckywhI68B2jSuyQ8HT1oe8xnKnRLjhWNb8RE8ONvZGHxFi3yE+50kMvFdeaXn1plfqy5aOFRAol2qB18fqCJpVieVvKIijjf/mbT2cyjIHkn+jktJPet2Us44p1Q7dqqVwJQbF05eapkptWb1aq6MwHuO5JfZL0C1JPEbysPJmOqG4MHZrZOAkzrQSnAksn5zeQ3SOvNr1UWrlSog0S7Vra4vyvOQu6Qn5x7ox5ET+eTv/Ae5W4YhycmktTnftq1A1UpGeirPDe3OuGs7MWXxFlqNXWir0jnqohiPnSE7SSSs1NzhkOpJpm/7RqUMqEBUTgJ2Kp1o2w2HeLtI26Ezv1Zd9MqhAhLtLDD4vNwtZ/DzR50oyatO0iW7OTz3UY7v2EDNjpdS/4q7SEqxz6JqpVoZ/c56vMXOg7ebQcnueWK9UvBRs3oyKqAWApwayCcN6uI60jl4lZBbWGQrzK3qYcfDcJzIVUogoaLMNZUXLRwqINEu1X3XF5+szuGPOpG7pRnVmxylw283cPvt1Xll5m6aXvsnqne41H+NJ1lAUWrGHjjL9v3jHzlZEFIwuB2UQpUfjYR0hxQduYXFljaUcHTzVio/O6yEX7y8e8pzkNaZX6smWjhUEAJnk3VNz5jAQTicWaBvFnmsoBr5OxpR98Is0uq/zbjf3Ev37hns27Obj77PKTOQAKXqMwuK2at2+/sRyitIIKxByW62O7hXBvPWZIdtbM4w723nvuskbtyuysKJ6QgU5tGUaXWLHqQ1sUQLhwpA8GwyJ8+LJ0mol+YhJ9cb1oCbnQ1blmTw1+vh6SVbKLnhFY4umczRgzs5e8ItQAY1a9Yks0dNS3VHYApvp9oLVmx3qNfspErx7U9P86AUzFq5i/Q0DynVkjiaZzx/3/aNWLb5EHtz8kj1JJXpm2+VE6kLpdtV2f+3d+5RVtXXHf/smbkDMzwcGB4hgDqAZiQpjjqxrkoV0CJaDJKaalxthpjWVVJXojUSolQlKRZDLT6aBugSY6rCqAmNlVhHAUk1hpe8xoKBVGyBEawFFbUEYfeP3+8O17n33MfcufecO7M/a911z+ve85095559fvv3++2d65wOyL9Mq2GEgTmHCJDqafTYCaW6suITieXSoQpLl8Itt7ikeQs+U0bbK//CWysXEavqy50/eIxx48a1H5/qZp3LU3FHguZTxM+VLpSSmIIjfsyhD49RFStn4TUNgYnxEvVPrB+ctrWRzbDcbAgKhdVUxejTqyLQ+eVTpjWqWGqQ7k0ozkFE9gDvA8eBj1W1UUQGAs3A6cAe4I9V9VAY+opNvh3Qa9fChAlu+eKL4Zqb32L2X/8p721fRe9R5zHoiptpbhtIw+Z9KW/E8Zt1Zx1DrMzNaQgimzH4Qcfc8sRWIDke3zGE0jC3JVD/8DTOI5tKeIlMrB+cMv351LOHBabUzrdMaxSxmdHdnzCHsk5U1QZVbfTrs4FVqnoGsMqv9wg6OwzxxAl44IGTjmHePFi9Gh7d8R9U1p3HgEl/zpCr76S8T03GXEiJldvSURUr508uOPUTwzOvOX9k+9DWVLl1gvoq9h3+qD0vT9Axx1UzzvTNVCnu5dmT+JurfidpWOl91zSw5c7JOd3M1ux8O6ftkP7/WKp1nW1mdPcnSmGlacAEv/wI8CLw7bDEFIKgZnhnhiG2tMDcufDLX0Lv3vDznx9j1aq5LFkygn2HR9LnrIuSPpMpF1K8ctsnaj+XCX17VwT2faR6gry5eQs3NW9p7yAOGn1UJmTVYsnUYZvuhiReY7ylke9NuDOtvKD/byk6hTg2M7r7E5ZzUKBFRBRYrKpLgKGq2gagqm0iMiTVB0XkBuAGgFNPPbVYerMmyAFk0wzPJn57/Dh873vOMQA8/DCcdtqLTJo0EYApX2qCUalrC8drD6RLFxHve8g2jhyUtC7xbwwalnpCs09/ke6mk26fkr7KXK50Zphxd5wLYDOjuz9hOYcLVXW/dwDPi8jObD/oHckSgMbGxsLMluok6RxAprh7Nk+1zz0Hd9wB69e79Y0bYd68L7JixQoAHnzwQZYdGQsBN8t41ChdSyXXp+tMT4rxcFW+E9vS3XQy1V/oyqfZzk42627DTMOadGcUj1Ccg6ru9+8HRWQFcD5wQESG+VbDMOBgGNryIZ0DyKcZfuwYnHkm7Nnj1pctg6lTP+CSSyax3nuKz/7Z33Hv3jqU4O+L1x7oyifZmuoYh9LUNIDU4aqqWDm9KspS9hV0HFmU6aaTqcBROseS64ib7tgK6Axmh+5P0Z2DiPQBylT1fb88Gfgu8DTQBMz37z8rtrZ8SecAOtsM37wZrr/+pGNobYXa2rcYP34K27ZtY/RZ4yj/wnc5ksW/MvFcXfEkG5SRtSNB4SpI7nOIT4KLz2nI5YY9919fS3JU6RxLZ0fcdLdWQGcxO3Rvwmg5DAVW+MpiFcDjqvpvIrIBeEJEvgb8F/ClELTlRToHkGsz/OhR6NfPtRpi/Y4yePp2xjQe5t6lL7H8h9+nqamJhQsXMmedZlXPIFYuKc+Vz1j1Bc+9njFBXjbhqsSZ4SJuEtyna6oC5zikInG+RLZ/T1hprg2jFBAtUJKzYtDY2KgbN24MW0Y7qWbCJo5KyfbGtW4dfPWrsGOHWx/xjRaOH9lF29IbAfjdiZex8snHqK2tpW72yozVygZUx7jzyuSx/Kn0xsqFPpUV7TOT091c050711QamWxXCIL0C+lnextGqSMimxKmEaQkSkNZS55McdhMzfB33oFBg9zyyJFw1owtfDh0H4dWP8R7G1ync+WnxlD2B7dSW1sLZF+oJhUpZ2Yf1/Z+gExhlnzOnY2WQj/F24gbwwjG6jl0MVedM5yXZ0/ijfl/mDJdcxBr18LgwSfXW1vhw0F7OLRmabtjGHTlrQxruo+2d08WyMknn342neHpJjZ1ZS7/MMbNWy0CwwjGnEPItLVBRYWb5VxXB48/7vIkHTiwi483NPPe+p/S9+wpjPjGMvqMvRhI7ljubEGZfJPN5XPubLUU8im+K/UbRnfD+hxCpKUFLrvMLU+YAM88A9XVypw5c7j//vvpXzuEyktvgsGj2z/TlXH4bLKFQufCRJnOm+3IJbtZG0bXk02fg7UcQmDvXujf3zmG+nrXWlizBg4d2ktZWRl333031dXVrHtpLffd+EcFe7Lt+OQ8oDpGrOyT+ZW6OswShZKahmFkxloOITBnjkuS19QEixa53EgrV65k6tSpAMRiMY4cOUJlZWXRtRU6DXNQkr2ubp0YhhGMjVaKKLNmwYwZMGaMW1+wYAG33XYbAPfccw+zZs0KTVuhJzZZwjbDKA3MOYRA//7utX//fmbOnElFRQXTp09n8eLFDBgwIGx5BaVUh49aYRujp2F9DiExc+ZMhg8fzgsvvMB1111Hc3Nzt3cMUJrDR4P6SdLVmDCMUsdaDiEwf/58Fi1aBMCmTZuor68PWVHxKMWEbZZmw+iJmHMIgfHjxzNlyhSWL1/OKaecEracolNqCdusn8ToiVhYKQTGjx/Ps88+2yMdQykSxgQ9wwgbcw6GkYFS7CcxjHyxsJJhZKAU+0kMI1/MORhGFpRaP4lh5IuFlQzDMIwkzDkYhmEYSZhzMAzDMJIw52AYhmEkYc7BMAzDSKKkU3aLyNvAm0U41SDgf4pwnq7AtBYG01oYTGvhSKf3NFUdHLAPKHHnUCxEZGOm3OdRwbQWBtNaGExr4chXr4WVDMMwjCTMORiGYRhJmHPIjiVhC8gB01oYTGthMK2FIy+91udgGIZhJGEtB8MwDCMJcw6GYRhGEuYc0iAiU0TkdRHZLSKzw9bTERHZIyLbRWSLiGz02waKyPMissu/h1aYWkSWishBEWlN2JZSnzge8LbeJiLnRkDrXSKyz9t3i4hckbDvO17r6yJyWRF1jhSRNSKyQ0ReE5Fv+u1RtWuQ3ijatreIrBeRrV7rXL+9TkTWeds2i0il397Lr+/2+0+PgNYficgbCXZt8Ntzvw5U1V4pXkA58BtgFFAJbAXGhq2rg8Y9wKAO274PzPbLs4F7QtR3EXAu0JpJH3AF8CwgwAXAughovQv4Vopjx/rroRdQ56+T8iLpHAac65f7Ab/2eqJq1yC9UbStAH39cgxY5232BHCt374ImOmXvw4s8svXAs1FtGuQ1h8BV6c4PufrwFoOwZwP7FbV/1TV3wLLgWkha8qGacAjfvkR4KqwhKjqL4D/7bA5SN804Mfq+BVQIyLDiqM0UGsQ04DlqnpUVd8AduOul4Kjqm2q+qpffh/YAQwnunYN0htEmLZVVT3iV2P+pcAk4Cm/vaNt4zZ/CrhERCRkrUHkfB2YcwhmOPDfCet7SX9Rh4ECLSKySURu8NuGqmobuB8mMCQ0dakJ0hdVe9/om+FLE0J0kdDqwxjn4J4aI2/XDnohgrYVkXIR2QIcBJ7HtVwOq+rHKfS0a/X73wVqw9KqqnG7zvN2XSgivTpq9WS0qzmHYFI9AURt3O+FqnoucDnwlyJyUdiC8iCK9v4hMBpoANqAe/320LWKSF/gJ8BNqvpeukNTbCu6XVPojaRtVfW4qjYAI3AtlrPS6ImUVhH5HPAdoB74PDAQ+LY/PGet5hyC2QuMTFgfAewPSUtKVHW/fz8IrMBdzAfizUX/fjA8hSkJ0hc5e6vqAf8DPAH8EyfDG6FqFZEY7kb7mKr+1G+OrF1T6Y2qbeOo6mHgRVx8vkZE4iWVE/W0a/X7TyH70GSXkaB1ig/jqaoeBR4mD7uacwhmA3CGH6lQietwejpkTe2ISB8R6RdfBiYDrTiNTf6wJuBn4SgMJEjf08BX/KiKC4B342GSsOgQk52Osy84rdf60Sp1wBnA+iJpEuAhYIeq/n3CrkjaNUhvRG07WERq/HIVcCmuj2QNcLU/rKNt4za/Glitvvc3JK07Ex4QBNc3kmjX3K6DYvWul+IL18P/a1zc8faw9XTQNgo3qmMr8FpcHy7muQrY5d8HhqhxGS5kcAz35PK1IH24Zu8PvK23A40R0PrPXss2/+MalnD87V7r68DlRdQ5HhcO2AZs8a8rImzXIL1RtO04YLPX1Arc4bePwjmo3cCTQC+/vbdf3+33j4qA1tXerq3Ao5wc0ZTzdWDpMwzDMIwkLKxkGIZhJGHOwTAMw0jCnINhGIaRhDkHwzAMIwlzDoZhGEYS5hyMHo2IHPfZK1tF5EkRqc7juyaIyDN++QuSJpOviNSIyNc7cY67RORbndVoGNlizsHo6Xykqg2q+jngt8BfJO70k4Zy/p2o6tOqOj/NITW4rJ6GEUnMORjGSf4dGCMip4urP/CPwKvASBGZLCKviMirvoXRF9prfuwUkZeAL8a/SERmiMg/+OWhIrLC597fKiK/B8wHRvtWywJ/3K0issEnTZub8F23i6tt8ALwmaJZw+jRmHMwDNpz41yOmz0K7ib8Y1U9B/gAmANcqi7R4Ubgr0SkNy4v0JXA7wOfCvj6B4C1qno2rmbEa7iaC7/xrZZbRWQyLlXE+bhkdOeJyEUich4udcs5OOfz+S7+0w0jJRWZDzGMbk2VT3sMruXwEPBp4E11ee/BJV8bC7zs0/VXAq/gsl++oaq7AETkUeAGkpkEfAVcJk3gXUmu0DfZvzb79b44Z9EPWKGqH/pzRCa/l9G9Medg9HQ+Upf2uB3vAD5I3ITLl//lDsc10HUpmgX4W1Vd3OEcN3XhOQwjayysZBiZ+RVwoYiMARCRahE5E9gJ1InIaH/clwM+vwqY6T9bLiL9gfdxrYI4zwHXJ/RlDBeRIcAvgOkiUuWz8F7ZxX+bYaTEnINhZEBV3wZmAMtEZBvOWdSr6v/hwkgrfYf0mwFf8U1goohsBzYBn1XVd3BhqlYRWaCqLcDjwCv+uKeAfupKbDbjspn+BBf6MoyCY1lZDcMwjCSs5WAYhmEkYc7BMAzDSMKcg2EYhpGEOQfDMAwjCXMOhmEYRhLmHAzDMIwkzDkYhmEYSfw/2OhBuBCA/RkAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
### Exercise
%% Cell type:markdown id: tags:
Use the `sklearn` function `mean_squared_error` to calculate the root mean squared error for this data.
%% Cell type:code id: tags:
``` python
from sklearn.metrics import mean_squared_error
print('Mean squared error is',np.round(mean_squared_error(Y_test,preds_linear),2))
```
%%%% Output: stream
Mean squared error is 3121.49
%% Cell type:markdown id: tags:
## Analysis of Coefficients
%% Cell type:markdown id: tags:
We can investigate the model coefficients to get an idea of the importance of each independent variable in predicting `Y`. However, we cannot directly compare coefficients as their dimensions are different. For example, the `AGE` coefficient will be inversely proportional to years, while the `BP` coefficient will be inversely proportional to mm of Hg. Hence to compare the coefficients we need to make them dimensionless and we also need to take into account the variability of each of the dependent variables. The variability of the features can be obtained from the standard deviation, which also has the same dimensions as the original variables.
To compare the importance of the coefficients it is easiest to normalise the independent variables from the start. The dependent variable does not need to be normalised. We normalise all the variables such that their mean is zero and their standard deviation is one, which can be done by subtracting the mean and then dividing by the standard deviation. This is equivalent to multiplying the coefficients by the standard deviation.
Normalising the coefficients is particularly important for regularization methods such as `Ridge` and `Lasso`, so that coefficients are treated uniformly by the penalty schemes.
%% Cell type:code id: tags:
``` python
nX =(X-X.mean())/X.std() # create nX, a normalised version of X
nX.describe() # show the descriptive statistics of nX
```
%%%% Output: execute_result
AGE SEX BMI BP S1 \
count 4.420000e+02 4.420000e+02 4.420000e+02 4.420000e+02 4.420000e+02
mean -3.717489e-17 1.321216e-16 -4.609686e-15 -9.625282e-16 -3.174936e-16
std 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00
min -2.251738e+00 -9.374744e-01 -1.895781e+00 -2.360375e+00 -2.662394e+00
25% -7.832846e-01 -9.374744e-01 -7.188104e-01 -7.697777e-01 -7.192046e-01
50% 1.130443e-01 -9.374744e-01 -1.529591e-01 -1.190789e-01 -9.073818e-02
75% 7.995940e-01 1.064282e+00 6.562083e-01 7.485196e-01 5.955183e-01
max 2.325260e+00 1.064282e+00 3.581660e+00 2.772916e+00 3.232188e+00
S2 S3 S4 S5 S6
count 4.420000e+02 4.420000e+02 4.420000e+02 4.420000e+02 4.420000e+02
mean 8.386956e-16 -1.446806e-16 -1.708035e-16 1.904051e-15 2.364248e-16
std 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00
min -2.427874e+00 -2.148448e+00 -1.604285e+00 -2.648040e+00 -2.893112e+00
25% -6.375263e-01 -7.374604e-01 -8.293610e-01 -6.981574e-01 -6.967595e-01
50% -8.020037e-02 -1.382738e-01 -5.443750e-02 -4.089059e-02 -2.263165e-02
75% 6.267323e-01 6.155415e-01 7.204860e-01 6.810788e-01 5.862581e-01
max 4.174548e+00 3.804760e+00 3.889923e+00 2.805543e+00 2.847848e+00
%% Cell type:markdown id: tags:
We repeat the linear regression, but now using the normalized variables. Again the square of the correlation coefficient is around 0.45.
%% Cell type:code id: tags:
``` python
# split into a training set with 80% of the data, and a testing set as the remainder
X_train, X_test, Y_train, Y_test = train_test_split(nX,Y,test_size=0.8, random_state=42)
linear = LinearRegression() # instantatiate the linear regression model
linear.fit(X_train,Y_train) # fit the data to the model
training_score = linear.score(X_train,Y_train) # calculate rsq for the training set
# use the independent variables for the testing set to predict the target variable
preds_linear = linear.predict(X_test)
# calculate the correlation of the predicted and actual target variables
rsquared_linear = r2_score(Y_test,preds_linear)
# print the training and testing scores
print("Training score is",round(training_score,4))
print("Testing score is",round(rsquared_linear,4))
```
%%%% Output: stream
Training score is 0.4689
Testing score is 0.4351
%% Cell type:markdown id: tags:
Now we can plot the linear regression coefficients. It can now be seen that `AGE`, `SEX`, `S3`, `S4` and `S6` appear to have the weakest effect on the prediction of Diabetes onset.
%% Cell type:code id: tags:
``` python
feature_names = X.columns.tolist() # write the column names to a list
# create a new dataframe with the regression coefficients from the normalised data
ncoefs = pd.DataFrame(linear.coef_.transpose(),columns=['Normalised'],index=feature_names)
# add our original coefficient importance to this dataframe
ncoefs.columns =['Normalised Coefficients']
# do a similar horizontal plot as before
ax = ncoefs.plot(kind='bar',figsize=(10,5))
plt.title('Linear Regression')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdQAAAFKCAYAAACkS65fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfZyVdZ3/8ddHICjRvAHUvAkyc01uRgXTVDJxFbNMM0srwSzxJkrLn6m1bXS3kTeZrJppmba1iiWWq2VZsVKmJrpoKK6GkkGKo64brEncfH5/nDN4GGdghvkO5xx8PR+Pecx1+70+58yZ8z7XdX2v60RmIkmSemaTehcgSdLGwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1XqQEQcEBH/Xe86NgYRsVNELI2IPvWuRepNBqpe0SJiQUQc3H56Zv4mM3etR03tRcSUiFheDaXnI+J3EbFvvevqqsx8IjMHZubKetci9SYDVWogEdG3k1nTM3MgMAiYCfxwA29f0joYqFIHIuLAiFhYM74gIv5fRDwQEf8bEdMjYkDN/HdGxJyaPciRNfPOiYj5EbEkIh6KiKNq5p0QEXdExEUR8RwwZW11ZeYK4AfA9hExuIvb3zMi/qu6/R9Wa/9y7eOMiLMj4ingu11o7+yIWFRt778jYlx1+t4RMTsi/hoRiyPi69XpQyMi28I6Il4XETdFxHMR8ceIOKmm7SkRcX1EfK/a/oMRMbrLfzipjgxUqeveB4wHhgEjgROgEljAVcDJwNbAt4CbIqJ/db35wAHAa4EvAN+PiO1q2n0L8BgwBPjK2gqIiFcBE4Bngf9Z1/ary98IXA1sBVwLHNWu2W2r814PTFpHe7sCk4ExmbkZcCiwoNrOxcDFmbk5sDNwfScP41pgIfA64L3Av7SFctURwHXAFsBNwCVre06kRmGgSl03LTP/kpnPAf8BtFSnnwR8KzPvzsyVmXkNsAzYByAzf1hdb1VmTgceBfauafcvmfmvmbkiM//WybbfFxHPA3+rbu+91b3VdW1/H6BvtfblmTkD+H27tlcBn8/MZdXtr629lUB/4M0R0S8zF2Tm/Go7y4E3RsSgzFyamXe1fxARsSOwP3B2Zr6YmXOAbwPH1yz228z8afWc678Bozp5TqSGYqBKXfdUzfALwMDq8OuBM6uHR5+vBt+OVPbAiIgJNYdPnweGUzkX2ubPXdj29Zm5BbANMBfYq2be2rb/OmBRrvktGO2315qZL3alvcz8I3AGlUPTT0fEdRHxuup6HwHeBDwcEfdExDs7eByvA57LzCU10/4EbF8z3v55HuC5XTUDA1XquT8DX8nMLWp+XpOZ10bE64ErqRwm3boainOBqFm/y1/5lJnPUDkUO6XmsHGn2weepHK+tXZ7O7ZvtquPp1rDv2fm/lSCN4GvVac/mpnHUTl0/TXgRxGxabu2/wJsFRGb1UzbCVjU1edAalQGqgT9ImJAzU9394auBE6JiLdExaYRcXg1NDalEjqtABHxYSp7qOstMx8Gfg58ugvbv5PKYdrJEdE3It7Nmoebu/V4ImLXiDioen74RSqHoFdWH9uHImJwZq4Cnq+2tcalMpn5Z+B3wFerz/VIKnu2P+jJcyI1AgNVgp9SCYa2nyndWTkzZ1M573gJlY5Cf6TaYSkzHwIupBJsi4ERwB0Faj6fSgeiIevY/t+B91AJreeBDwE3Uzkn2u3HQ+X86VTgGSqHZocAn6nOGw88GBFLqXRQOrbdoeQ2xwFDqeyt3kjl/O1t3Xz8UsMJv2BcemWJiLuByzPzu/WuRdqYuIcqbeQi4m0RsW31kO9EKpf83FrvuqSNjT3npI3frlSuCR1I5ZrY92bmk/UtSdr4eMhXkqQCPOQrSVIBBqokSQXU5RzqoEGDcujQofXYtCRJ6+3ee+99JjMHdzSvLoE6dOhQZs+eXY9NS5K03iLiT53N85CvJEkFGKiSJBVgoEqSVIA3dpDUVJYvX87ChQt58cWObhMslTFgwAB22GEH+vXr1+V1DFRJTWXhwoVsttlmDB06lDW/lU4qIzN59tlnWbhwIcOGDevyel0+5Fv9qqXfR8T9EfFgRHyhOn1YRNwdEY9GxPSIeNV61C9JXfLiiy+y9dZbG6bqNRHB1ltv3e2jIN05h7oMOCgzRwEtwPiI2IfKFwlflJm7UPmqp490qwJJ6ibDVL1tfV5jXQ7UrFhaHe1X/UngIOBH1enXAEd2uwpJaiIRwZlnnrl6/IILLmDKlCkbtIYTTjiBH/2o8tb70Y9+lIceeqjHbQ4cOLDD6U899RTHHnssO++8M29+85t5xzvewSOPPLJe25g2bRq77bYbH/zgB1m2bBkHH3wwLS0tTJ8+fZ2P46abbmLq1Knrtd3nn3+eyy67bL3W7apunUONiD7AvcAbgUupfHPF85m5orrIQmD7TtadBEwC2Gmnnda3Xklaw9Bzbina3oKph69zmf79+zNjxgzOPfdcBg0a1O1trFixgr59y3Vh+fa3v12srfYyk6OOOoqJEydy3XXXATBnzhwWL17Mm970pm63d9lll/Gzn/2MYcOGcdddd7F8+XLmzJkDwPvf//61rnvEEUdwxBFHdP9B8FKgnnbaaeu1fld067KZzFyZmS3ADsDewG4dLdbJuldk5ujMHD14cId3bZKkptC3b18mTZrERRdd9LJ5f/rTnxg3bhwjR45k3LhxPPHEE0Blj/JTn/oUb3/72zn77LOZMmUKEydO5JBDDmHo0KHMmDGDT3/604wYMYLx48ezfPlyAL74xS8yZswYhg8fzqRJk+joG8IOPPBAZs+ezcqVKznhhBMYPnw4I0aMWF3f/PnzGT9+PHvttRcHHHAADz/8MACPP/44++67L2PGjOFzn/tch4915syZ9OvXj1NOOWX1tJaWFg444AAyk7POOmv19qZPn756mfPPP58xY8YwcuRIPv/5zwNwyimn8Nhjj3HEEUfwta99jQ996EPMmTOHlpYW5s+fv/pxANx6663sueeejBo1inHjxgFw9dVXM3nyZABaW1s5+uijGTNmDGPGjOGOO+4AYMqUKZx44okceOCBvOENb2DatGkAnHPOOcyfP5+WlhbOOussnnzyScaOHUtLSwvDhw/nN7/5TZf+9muzXh+RMvP5iPhPYB9gi4joW91L3QH4S4+rUq8r/am+TVc+3Usbg4997GOMHDmST3/602tMnzx5MhMmTGDixIlcddVVfOITn+DHP/4xAI888gi//OUv6dOnD1OmTGH+/PnMnDmThx56iH333ZcbbriB8847j6OOOopbbrmFI488ksmTJ/PP//zPABx//PHcfPPNvOtd7+qwpjlz5rBo0SLmzp0LVPbKACZNmsTll1/OLrvswt13381pp53Gr3/9a04//XROPfVUJkyYwKWXXtphm3PnzmWvvfbqcN6MGTOYM2cO999/P8888wxjxoxh7Nix/OEPf+DRRx/l97//PZnJEUccwaxZs7j88su59dZbmTlzJoMGDeItb3kLF1xwATfffPMa7ba2tnLSSScxa9Yshg0bxnPPPfeybZ9++ul88pOfZP/99+eJJ57g0EMPZd68eQA8/PDDzJw5kyVLlrDrrrty6qmnMnXqVObOnbt6b/jCCy/k0EMP5bOf/SwrV67khRde6PAxdkeXAzUiBgPLq2H6auBgKh2SZgLvBa4DJgI/6XFVktTgNt98cyZMmMC0adN49atfvXr6nXfeyYwZM4BKANYG7jHHHEOfPn1Wjx922GH069ePESNGsHLlSsaPHw/AiBEjWLBgAVDZQzzvvPN44YUXeO6559h99907DdQ3vOENPPbYY3z84x/n8MMP55BDDmHp0qX87ne/45hjjlm93LJlywC44447uOGGG1bXevbZZ3frOfjtb3/LcccdR58+fdhmm21429vexj333MOsWbP4xS9+wR577AHA0qVLefTRRxk7dmyX2r3rrrsYO3bs6ktWttpqq5ct88tf/nKN861//etfWbJkCQCHH344/fv3p3///gwZMoTFixe/bP0xY8Zw4oknsnz5co488khaWlq69dg70p091O2Aa6rnUTcBrs/MmyPiIeC6iPgy8F/Ad3pclSQ1gTPOOIM999yTD3/4w50uU9tbdNNNN11jXv/+/QHYZJNN6Nev3+plN9lkE1asWMGLL77IaaedxuzZs9lxxx2ZMmXKWi/l2HLLLbn//vv5+c9/zqWXXsr111/PN77xDbbYYovVe2Zrq68ju+++++rOT+11dPi5bfq5557LySefvNa2O5OZ66xr1apV3HnnnWt8mGnT9rwC9OnThxUrVrxsmbFjxzJr1ixuueUWjj/+eM466ywmTJiwXvW26U4v3wcyc4/MHJmZwzPzi9Xpj2Xm3pn5xsw8JjOX9agiSWoSW221Fe973/v4znde2o9461vfurrzzg9+8AP233//9W6/LTwHDRrE0qVLOw22Ns888wyrVq3i6KOP5ktf+hL33Xcfm2++OcOGDeOHP/whUAmr+++/H4D99ttvjVo7ctBBB7Fs2TKuvPLK1dPuuecebr/9dsaOHcv06dNZuXIlra2tzJo1i7333ptDDz2Uq666iqVLKxeGLFq0iKeffrrLj3vffffl9ttv5/HHHwfo8JDvIYccwiWXXLJ6vLMPDG0222yz1XuwUDnXPWTIEE466SQ+8pGPcN9993W5vs54pyRJ6oEzzzxzjTf2adOmceKJJ3L++eczePBgvvvd765321tssQUnnXQSI0aMYOjQoYwZM2atyy9atIgPf/jDrFq1CoCvfvWrQCUsTz31VL785S+zfPlyjj32WEaNGsXFF1/MBz7wAS6++GKOPvroDtuMCG688UbOOOMMpk6dyoABAxg6dCjf+MY3GDt2LHfeeSejRo0iIjjvvPPYdttt2XbbbZk3bx777rsvULkc5/vf/z5Dhgzp0uMePHgwV1xxBe95z3tYtWoVQ4YM4bbbbltjmWnTpq0+j71ixQrGjh3L5Zdf3mmbW2+9Nfvttx/Dhw/nsMMOY/jw4Zx//vn069ePgQMH8r3vfa9Lta1NdLbL3ptGjx6dfh9qfdkpSc1q3rx57LZbRxcYSGV19FqLiHszc3RHy/ttM5IkFWCgSpJUgIEqSVIBBqqkplOPvh96ZVmf15iBKqmpDBgwgGeffdZQVa9p+z7UAQMGdGs9L5uR1FR22GEHFi5cSGtra71L0UZswIAB7LDDDt1ax0CV1FT69eu3+pZ0UiPxkK8kSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUQJcDNSJ2jIiZETEvIh6MiNOr06dExKKImFP9eUfvlStJUmPqzheMrwDOzMz7ImIz4N6IuK0676LMvKB8eZIkNYcuB2pmPgk8WR1eEhHzgO17qzCpzdBzbumVdhdMPbxX2pX0yrRe51AjYiiwB3B3ddLkiHggIq6KiC07WWdSRMyOiNmtra3rVawkSY2q24EaEQOBG4AzMvOvwDeBnYEWKnuwF3a0XmZekZmjM3P04MGDe1CyJEmNp1uBGhH9qITpDzJzBkBmLs7MlZm5CrgS2Lt8mZIkNbbu9PIN4DvAvMz8es307WoWOwqYW648SZKaQ3d6+e4HHA/8ISLmVKd9BjguIlqABBYAJxetUJKkJtCdXr6/BaKDWT8tV44kSc3JOyVJklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQV0OVAjYgdI2JmRMyLiAcj4vTq9K0i4raIeLT6e8veK1eSpMbUnT3UFcCZmbkbsA/wsYh4M3AO8KvM3AX4VXVckqRXlC4HamY+mZn3VYeXAPOA7YF3A9dUF7sGOLJ0kZIkNbr1OocaEUOBPYC7gW0y80mohC4wpJN1JkXE7IiY3draun7VSpLUoLodqBExELgBOCMz/9rV9TLziswcnZmjBw8e3N3NSpLU0LoVqBHRj0qY/iAzZ1QnL46I7arztwOeLluiJEmNrzu9fAP4DjAvM79eM+smYGJ1eCLwk3LlSZLUHPp2Y9n9gOOBP0TEnOq0zwBTgesj4iPAE8AxZUuUJKnxdTlQM/O3QHQye1yZciRJak7eKUmSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKmALgdqRFwVEU9HxNyaaVMiYlFEzKn+vKN3ypQkqbF1Zw/1amB8B9MvysyW6s9Py5QlSVJz6XKgZuYs4LlerEWSpKZV4hzq5Ih4oHpIeMvOFoqISRExOyJmt7a2FtisJEmNo6eB+k1gZ6AFeBK4sLMFM/OKzBydmaMHDx7cw81KktRYehSombk4M1dm5irgSmDvMmVJktRcehSoEbFdzehRwNzOlpUkaWPWt6sLRsS1wIHAoIhYCHweODAiWoAEFgAn90KNkiQ1vC4HamYe18Hk7xSsRZKkpuWdkiRJKsBAlSSpAANVkqQCDFRJkgowUCVJKsBAlSSpgC5fNiNJ2vgMPeeWXml3wdTDe6XdRuYeqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBXQ5UCPiqoh4OiLm1kzbKiJui4hHq7+37J0yJUlqbN3ZQ70aGN9u2jnArzJzF+BX1XFJkl5xuhyomTkLeK7d5HcD11SHrwGOLFSXJElNpafnULfJzCcBqr+H9LwkSZKazwbrlBQRkyJidkTMbm1t3VCblSRpg+hpoC6OiO0Aqr+f7mzBzLwiM0dn5ujBgwf3cLOSJDWWngbqTcDE6vBE4Cc9bE+SpKbUnctmrgXuBHaNiIUR8RFgKvCPEfEo8I/VcUmSXnH6dnXBzDyuk1njCtUiSVLT8k5JkiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBfQt0UhELACWACuBFZk5ukS7kiQ1iyKBWvX2zHymYHuSJDUND/lKklRAqUBN4BcRcW9ETCrUpiRJTaPUId/9MvMvETEEuC0iHs7MWbULVIN2EsBOO+1UaLOSJDWGInuomfmX6u+ngRuBvTtY5orMHJ2ZowcPHlxis5IkNYweB2pEbBoRm7UNA4cAc3variRJzaTEId9tgBsjoq29f8/MWwu0K0lS0+hxoGbmY8CoArVIktS0vGxGkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCSt4cXxIw9Jxbire5YOrhxduUVJZ7qJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUQJFAjYjxEfHfEfHHiDinRJuSJDWTHgdqRPQBLgUOA94MHBcRb+5pu5IkNZMSe6h7A3/MzMcy8+/AdcC7C7QrSVLT6Fugje2BP9eMLwTesrYVnn32Wa6++uoCm9b6Gv+qZ3ul3auvbi3eZjPVCr1Tb2/VKjXb/1cjKxGo0cG0fNlCEZOASQDbb799lxu/67Hyf+x93rB18Tahd2qF3qm3t56D3tBMtUJz1ev/V+/U20y1NtPrFRr7NRuZL8u+7jUQsS8wJTMPrY6fC5CZX+1sndGjR+fs2bO71P7Qc27pUX0dWTD18OJtQu/UCr1Xr+T/V+/U20y1Npt6v2Yj4t7MHN3RvBLnUO8BdomIYRHxKuBY4KYC7UqS1DR6fMg3M1dExGTg50Af4KrMfLDHlUmS1ERKnEMlM38K/LREW5IkNSPvlCRJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRS59aAkSRtCI3/jjnuokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBXhjB0kqrJFvPqDeY6BKr2C+8UvlNHyg+g8vSWoGnkOVJKkAA1WSpAJ6FKgRMSUiFkXEnOrPO0oVJklSMylxDvWizLygQDuSJDUtD/lKklRAiUCdHBEPRMRVEbFlgfYkSWo66wzUiPhlRMzt4OfdwDeBnYEW4EngwrW0MykiZkfE7NbW1mIPQJKkRrDOc6iZeXBXGoqIK4Gb19LOFcAVAKNHj86uFihJUjPoaS/f7WpGjwLm9qwcSZKaU097+Z4XES1AAguAk3tckSRJTahHgZqZx5cqRJKkZuZlM5IkFWCgSpJUgIEqSVIBBqokSQUYqJIkFdDwXzDeTPwydEl65XIPVZKkAgxUSZIKMFAlSSrAQJUkqQADVZKkAgxUSZIKMFAlSSrA61AlNQWv81ajcw9VkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCDFRJkgowUCVJKiAyc8NvNKIV+FMvND0IeKYX2u0N1to7rLX3NFO91to7rBVen5mDO5pRl0DtLRExOzNH17uOrrDW3mGtvaeZ6rXW3mGta+chX0mSCjBQJUkqYGML1CvqXUA3WGvvsNbe00z1WmvvsNa12KjOoUqSVC8b2x6qJEl1YaBKklSAgSpJUgF9613A+oiIf8jMh6vD/TNzWc28fTLzrvpVt6aI+FfgM5m5pN30fwAuycyD61NZxyLiU2ubn5lf31C1dEVEHAm8EfhDZv683vVow4qI1wCTgQT+FTgWeA/wMPDFzFxax/K6JCIeycw31bsO9Vyz7qH+e83wne3mXbYhC+mCp4A5EfEBqLwBRMR5wE3ApXWtrGObreOnYUTEZcAnga2BL0XE5+pc0nqJiIbqORkRfSLi5Ij4UkTs127eP9Wrrk5cDWwDDANuAUYDFwABfLN+ZXUsIpZExF+rP0siYgmwc9v0etdXKyJG1gz3i4h/ioibIuJfqh9kGkZETI6IQdXhN0bErIh4PiLujogRG6yOZuzlGxH/lZl7tB/uaLwRRMQw4BIqgfQ64Hrgy5n5Ql0La3IRMRcYlZkrq//gv8nMvepdV0ciYqvOZgH3Z+YOG7KetYmIbwOvAX4PHA/cnpmfqs67LzP3rGd9tSJiTma2REQATwLbZWZWx+/PzJHraGKDqh6xei1wVmYurk57PDOH1beyl6v9W0fEhVQ+uH4XOBLYOjMn1LO+WhHxYGbuXh2+Bfh2Zt4YEQcCX8nM/dbaQCFNeciXyuGdjoY7Gm8EbTX1pXJUYF6jhmlETFvb/Mz8xIaqpQv+npkrATLzheqbaKNqu391bY1ZHR9Sl4o6t3dbEEXEJcBlETEDOI41628Y1RD9aVb3EKrjDfdekJkfj4i9gGsj4sdUPmg3XJ1VtX/rccCYzFweEbOA++tUU2dqs2xIZt4IkJn/GREb7MhaswbqDtU3/qgZpjq+ff3KernqIbITgM9m5vSI2B64OCI+CpyamQ/VtcCXOwWYS2Uv+i806Bto1T9ExAPV4aBy6OyB6nA22N7JY8C4zHyi/YyI+HMd6lmbV7UNZOYKYFJEfB74NTCwblV1bHZEDMzMpZl5YtvEiNgZWLKW9eomM++NiIOpnPu9HRhQ55I689qIOIrKTkD/zFwODfth5UcRcTXwReDGiDgDmEHlg8DL/ud6S7MG6lk1w7PbzWs/Xm+DgT3aOiVl5iLgvRFxGHADsFs9i+vAdsAxwPuBFcB04IbM/J+6VtWxRnvu1uYbwJZ0/M993gauZV1mR8T4zLy1bUJmfiEiFtF45yW/RSXklwJExATgaCrP85F1rKtDETEG+HNmPgVMi4hNgS9ExMXAFzLzufpWuIbbgSOqw3dFxDaZuTgitqXBvnEmMz8bEScA1wI7A/2BScCPgQ9uqDqa8hzqxqJ9D+VGU92bPg74FHB2Zv5bnUtap2rHhGezwV7Y7d5Ia9/4/wRMaaQ30mqtCzPzyep4I9d6H3BwZj4XEWOB64CPAy3Abpn53roW2E4z1dtkr4P2/18TqdS6gA1Ya1P28o2I/at/3LbxH0XEr6s/B9WztvYi4vqa4a+1m/0fG7icLouIPYEzgA8BPwPurW9FLxcR+0TEf0bEjIjYo9pJaS6wOCLG17u+dr4F/B2g+kY6Ffge8L803v1RvwUsg6aotU/Nm+X7gSsy84bM/ByVy6kaTTPV20yvg/b/X18FrmED19qsh3y/QOVTXZtdqZyn3BT4DJVzPY1il5rhfwTOrhnv8Etq6ykivgC8E5hH5dPzudXzaI3oEip/79dS+Zsflpl3ReUa32uBW9e28gbW4RspcENEzKljXR1pqlojom/1NTqOymG+No34/tZM9TbV66ARam3KPVRg83adeR7NzHszcxYNdq0ka+/B11CHJas+RyWgRlH5lHdfRDwQEX+o6QDUKPpm5i8y84fAU2039Gi76UeD6RMRbW+Y41jzQ1/DvZE2Ua3XArdHxE+AvwG/gcq1iFT2ThpNM9XbTK+Dhqi10Z6UrtqidiQz31Mzus0GrmVdXhMRe1D58PLq6qFUqPREfXX9yupUw10Ptxaraob/1m5eo31YaXsjfYbGfyNtmloz8ysR8Ssqnel+UXPufBPWPIrVEJqs3qZ5HdAgtTZlp6SI+A/g8sy8pd30d1K5FOXw+lT2chExk5euN4Q13+gjM9++4avqngbu6LMS+D9e+nDSdpzz3kEAAAGYSURBVG1vAAMys1+9autIROzDS2+k/1ed9iZgYGbeV9fi2mmmWtV7mul10Ai1NmugvpHKbcZ+B7Q9UXsBbwXemZmP1Ku29iJibyq9z9p6ytWl91lXVV+UU4HngC8B/wYMovIJekLtpRSSpJc0ZaBC5ZITKtcX7U5lr+9B4FHguMz8WD1rq9VM3eQBImI2L3X0uYJ2HX2ywW7rKEmNomkDtU31/ORxwPuAx6nchOCS+lb1koi4PzNHVYcvBVozc0p1fE5mttSzvvZqa4qIeZm5W828hrtPsiQ1iqbslFQ9Ln4slSB9lsrdfBr1fGQzdZOH5uroI0kNoxHf0LviYSq9uN6VmX8EiIhP1rekTjVE77NuGBWVr5EKKr2S275SKmjce45KUt015SHf6g2bj6XSCelWKuclv50N+BVI0Bi9zyRJvaspA7VN9cbSR1I59HsQlVtN3ZiZv6hrYZKkV5ymDtRaUfkC52OA92dmQ93PV5K08dtoAlWSpHpq1nv5SpLUUAxUSZIKMFAlSSrAQJUkqQADVZKkAv4/7FIMsFqiJEcAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
However, we need to consider the variability of the coefficients as the training set is varied. This gives an indication of the variance or overfitting of the model.
To investigate the variability we can use the `sklearn` methods `cross_validate` and `RepeatedKFold`. The first of these performs a number of runs of a model. The second splits the data in n sections and repeats the calculations m times. This gives n.m runs to investigate the variability of the coefficients. The variability of these can then be plotted using a box and whisker plot.
We can see `AGE`, `SEX`, `BP`, `BMI` and `S6` have very low variance, whereas `S1`-`S5` have high variance indicative of overfitting. Of the low variance features, `AGE` and `S6` seem to have little effect on the predictions.
%% Cell type:code id: tags:
``` python
from sklearn.model_selection import cross_validate, RepeatedKFold # import sklearn methods
rng = np.random.RandomState(1) # make sure the results are repeatable
feature_names = nX.columns.tolist() # write the column names to a list
# cross_validate takes the particular model, in this case linear regression which we
# instantatiated earlier,
# and undertakes a number of runs according the method specified by cv=
# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times,
# giving 25 runs
# return_estimator=True returns the fitting data for each run
scores = cross_validate(linear, nX, Y, cv=RepeatedKFold(n_splits=5, n_repeats=5),
return_estimator=True,return_train_score=True)
# take the results for each simulation (estimator), extract the coefficients for each run
# and add them to a dataframe with columns being the feature names
coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)
# plot the descriptive statics of the coefficients in a box and whisker plot to show
# variability
ax = coefs.plot(kind='box',figsize=(10,5))
plt.title('Linear Regression')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
print('Testing score is',
round(np.mean(scores['test_score']),4),"±",round(np.std(scores['test_score']),4))
```
%%%% Output: stream
Testing score is 0.4862 ± 0.0519
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZ/UlEQVR4nO3dfbRddX3n8fe3IUIUDBVCgQC5jFIIDWLtLeBIh/pQxMKAdNFKtApOWkZHmHZGhwSzZoC2TMNi1YdCnYoCAcsEEKVSQIvtRGtUtBfkOYhBI88aQCKMwbmE7/yx99WTm3NuzuWe393n3Lxfa53FOXvv89vfuzk5n7N/v/0QmYkkSSrjl5ouQJKkmcyglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWmoSI+K2I+E7TdcwEEbFfRDwbEbOarkUqyaCV2oiI9RHx5vHTM/OrmXlgEzWNFxHnRMRoHVZPR8TXI+J1TdfVrcx8MDN3zszNTdcilWTQSgMgInboMOvqzNwZ2B1YDXxmmtcvaRsMWmkSIuK3I+LhltfrI+KDEXFnRGyMiKsjYqeW+cdFxO0te5yvbpm3LCIeiIhnIuLeiDixZd6pEfG1iPhIRDwFnDNRXZn5PHAlMD8i5nW5/tdGxLfr9X+mrv0vWv/OiFgaEY8Dl3XR3tKIeKRu7zsR8aZ6+mERMRIRP4mIH0bEh+vpQxGRYyEeEXtHxPUR8VRErIuIP25p+5yIuCYirqjbvycihrv+Hyc1yKCVpu4PgGOA/YFXA6dCFWTApcB/BHYDPgFcHxE71u97APgtYC5wLvB3EbFXS7uHA98D9gDOm6iAiHgJ8G7gSeDH21p/vfx1wErgFcAq4MRxze5Zz1sAnLaN9g4ETgd+MzN3Ad4CrK/b+Rjwscx8OfBK4JoOf8Yq4GFgb+Ak4H+OhXXteOAqYFfgeuCiibaJ1C8MWmnq/jozH83Mp4B/AF5TT/9j4BOZ+c3M3JyZlwM/A44AyMzP1O97ITOvBr4LHNbS7qOZeWFmPp+Zmzqs+w8i4mlgU72+k+q9222t/whgh7r20cz8HPCtcW2/AJydmT+r1z9Re5uBHYGDI2J2Zq7PzAfqdkaBV0XE7pn5bGbeMv6PiIh9gSOBpZn5XGbeDnwKeFfLYmsy86Z6TPfTwKEdtonUVwxaaeoeb3n+U2Dn+vkC4AN1N+vTdSDuS7XHRkS8u6Ub9mlgEdVY65iHulj3NZm5K/ArwN3Ab7TMm2j9ewOP5JZ3FRm/vg2Z+Vw37WXmOuBPqbq4fxQRV0XE3vX7lgC/CtwXEf8aEce1+Tv2Bp7KzGdapv0AmN/yevx23smxYw0Cg1Yq5yHgvMzcteXx0sxcFRELgE9SdbfuVofl3UC0vL/rW2tl5hNUXbrntHQ/d1w/8BjVeG7r+vYd32y3f09dw//OzCOpAjmB8+vp383MxVRd4OcD10bEy8a1/SjwiojYpWXafsAj3W4DqV8ZtFJnsyNip5bHZPeePgm8NyIOj8rLIuLYOkxeRhVGGwAi4j1Ue7QvWmbeB/wjcGYX6/8GVXfv6RGxQ0ScwJbd1pP6eyLiwIh4Yz3+/BxVV/bm+m/7w4iYl5kvAE/XbW1xSk9mPgR8HfjLelu/mmpP+MqpbBOpHxi0Umc3UQXG2OOcybw5M0eoxjUvojpAaR31gVKZeS/wV1SB90PgEOBrPaj5AqoDl/bYxvr/H/B7VGH2NPCHwA1UY66T/nuoxmdXAE9QdfHuAXyonncMcE9EPEt1YNTJ47qkxywGhqj2bq+jGh/+0iT/fqnvhDd+lwQQEd8E/jYzL2u6FmkmcY9W2k5FxFERsWfddXwK1alJX2y6Lmmm8Yg9aft1INU5rTtTndN7UmY+1mxJ0sxj17EkSQXZdSxJUkEGrSRJBfXVGO3uu++eQ0NDTZchSdKk3HrrrU9k5rx28/oqaIeGhhgZGWm6DEmSJiUiftBpnl3HkiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklRQX12CUZLU3yJiUst7K9Ye7tFGxKyI+HZE3FC/3j8ivhkR342IqyPiJb1alySpGZm51WPB0hvaTjdkK73sOv4TYG3L6/OBj2TmAcCPgSU9XJckSQOhJ0EbEfsAxwKfql8H8Ebg2nqRy4G39WJdkiQNkl7t0X4UOBN4oX69G/B0Zj5fv34YmN+jdUmSNDCmHLQRcRzwo8y8tXVym0XbdtZHxGkRMRIRIxs2bJhqOZIk9ZVe7NG+Hjg+ItYDV1F1GX8U2DUixo5q3gd4tN2bM/PizBzOzOF589renF6SpIE15aDNzLMyc5/MHAJOBv5PZr4TWA2cVC92CvD5qa5LkqRBU/KCFUuB/xoR66jGbC8puC5JkvpS9NN5TsPDwzkyMtJ0GZIk4NBzb2bjptGetjl3zmzuOPvonrbZDyLi1swcbjfPK0NJktrauGmU9SuO7WmbQ8tu7Gl7g8BrHUuSVJBBK0lSQQatJEkFGbSSJBXkwVCSpLZ2WbiMQy5f1uM2obo0/vbDoJUktfXM2hUeddwDdh1LklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkGe3iOpKxExqeX76c5gUpMMWkld6RScQ8tu7Pm5luofvT7vde6c2T1tbxAYtJKktrr9AeWPrYk5RitJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQV5Hm0kqSudbpCWJzffnmvEGbQSpImweCcPLuOJUkqyKCVJKkgg1aSpIIMWkmSCvJgKElbOfTcm9m4abTr5bu9ldrcObO54+yjX2xZ0kAyaCVtZeOm0SK3Pev1vU2lQWDQqiudzp3rxFMAJKniGK26kplbPRYsvaHtdENWkn7BPVptwbE5Seotg1ZbeGHoA+xSol0A7irQsiT1N4NWW3hm7QoPgpGkHjJoJW1ll4XLOOTyZQXaBej9Dzmpnxm0krZiz4bUOwattlLiy3DunNk9b1OSBoFBqy1MZi9maNmNRfZ6JGkm8TxaSZIKMmglSSrIoJUkqSCDVpKkgjwYSl3pdFOBOL/98l7vWJIqU96jjYh9I2J1RKyNiHsi4k/q6a+IiC9FxHfr//7y1MtVUzrdPMCbCkjSxHrRdfw88IHMXAgcAbw/Ig4GlgH/nJkHAP9cv5Ykabsy5aDNzMcy87b6+TPAWmA+cAJweb3Y5cDbprouSZIGTU8PhoqIIeDXgW8Cv5KZj0EVxsAevVyXJEmDoGdBGxE7A58F/jQzfzKJ950WESMRMbJhw4ZelSNJUl/oSdBGxGyqkL0yMz9XT/5hROxVz98L+FG792bmxZk5nJnD8+bN60U5kiT1jV4cdRzAJcDazPxwy6zrgVPq56cAn5/quiRJGjS9OI/29cC7gLsi4vZ62oeAFcA1EbEEeBD4/R6sS5KkgTLloM3MNUD7qxnAm6bavqRmeLtEqTe8MpRmnE5XserEC2xszdslSr3jtY4147S7UtWCpTd4FStJjTBoJUkqyKCVJKkgx2g10A4992Y2bhrtatluD+6ZO2c2d5x99FTKkqSfM2g10DZuGu35gTgljraVtP2y61iSpILco9VA22XhMg65vLd3YNxlIYCnq0jqDYNWA+2ZtSvsOpbU1wxaSWqYF1mZ2QxaDbxe74F6mUBNt3bB6RW3Zg6DVgOt2y8iv7QkNcWg1YzTqRsuzm+/vN1wkkoyaDXjGJyS+onn0UqSVJBBK0lSQQatJEkFGbSSJBXkwVCSujLRRRXaHdHtQWlSxaCV1BWDU3pxDFpJmiaTuX8yeA/lmcKglaRpUuL+yeCNMPqdB0NJklSQe7SSZqTJ3BFnusafS9w/uWoXvIdy/zJoJc1I/XhHnBL3Twa7jvudXceSJBVk0EqSVJBBK0lSQQatJEkFeTCUpIHmRSDUyWSOPIdyR58btJIGmheBUCedgnO6jz6361iSpIIMWkmSCjJoJUkqyDFaSQNt0C5rWGLsd+6c2T1vU71j0EoaaIN0WcPJ1Nn05SLVO3YdS5JUkHu0kqSBV+J86l6dS23QSpIGXonzqXs1fGDXsSRJBblHK0kaeCWOPu/VkecGraSB5ykzKnH0ea8+VzMqaPvlAtKSpo+nzKjfzagx2sxs+1iw9Ia20yVJKm1G7dFKkrZfvR5C6NXwQfGgjYhjgI8Bs4BPZeaK0uuUJG1f+nkIoWjXcUTMAv4GeCtwMLA4Ig4uuU5JkvpJ6T3aw4B1mfk9gIi4CjgBuHeqDZe4Cgj07kogkqRmTXSAbJy/9bRSx+6UDtr5wEMtrx8GDu+08JNPPsnKlSu7avh1m5/kiIN2m1Jx7dzyvSdZufLRnrcrqXnHvORJVq7c0HQZXRmkWvvVZZddNqnlu82fySodtO1+TmzxkyEiTgNOA5g/f37hciSp/3zlK19pP/2htpM56qijClajXouSp7lExOuAczLzLfXrswAy8y/bLT88PJwjIyNdtV1qMNvz7KSZy3/fKiUibs3M4XbzSu/R/itwQETsDzwCnAy8oxcND9rNniVJ26eiQZuZz0fE6cA/Up3ec2lm3tOLtgfpZs+SpO1X8fNoM/Mm4KYSbXt9U0lSvxvYK0P188nJkiSNGdigbadfzpmSJGnMjApag1OS1G9m1N17JEnqNzNqj1aSxnQaSnIYSdPNoJU0Ixme6hd2HUuSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0PaJVatWsWjRImbNmsWiRYtYtWpV0yVJknrAK0P1gVWrVrF8+XIuueQSjjzySNasWcOSJUsAWLx4ccPVSZKmIvrpMmXDw8M5MjLSdBnTbtGiRVx44YW84Q1v+Pm01atXc8YZZ3D33Xc3WJkkqRsRcWtmDredZ9A2b9asWTz33HPMnj3759NGR0fZaaed2Lx5c4OVSZK6MVHQOkbbBxYuXMiaNWu2mLZmzRoWLlzYUEWSpF4xaPvA8uXLWbJkCatXr2Z0dJTVq1ezZMkSli9f3nRpkqQp8mCoPjB2wNMZZ5zB2rVrWbhwIeedd54HQknSDOAYbYM63Zi6nX76/yRJ2pJjtH0qM7d6LFh6Q9vpkqTBZNBKklSQY7TT4NBzb2bjptGulx9admNXy82dM5s7zj76xZYlSZoGBu00eGHoA+xSol0A7irQsiSpVwzaafDM2hWsX3Fsz9vtds9XktQcx2glSSrIPdppUmLvc+6c2dteSJLUKIN2Gkym23ho2Y1FupklSc2w61iSpIIMWkmSCjJoJUkqyDHaBnW61nGcv/U0L8MoSYPJoG2Q4SlJM59dx5IkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJU0JSCNiIuiIj7IuLOiLguInZtmXdWRKyLiO9ExFumXqokSYNnqnu0XwIWZeargfuBswAi4mDgZODXgGOAj0fErCmuS5KkgTOloM3MmzPz+frlLcA+9fMTgKsy82eZ+X1gHXDYVNYlSdIg6uUY7X8AvlA/nw881DLv4XqaJEnblW3eJi8i/gnYs82s5Zn5+XqZ5cDzwJVjb2uzfNt7wkXEacBpAPvtt18XJUuSNDi2GbSZ+eaJ5kfEKcBxwJvyFzdYfRjYt2WxfYBHO7R/MXAxwPDwsDdolSTNKFM96vgYYClwfGb+tGXW9cDJEbFjROwPHAB8ayrrkiRpEG1zj3YbLgJ2BL4UEQC3ZOZ7M/OeiLgGuJeqS/n9mbl5iuuSJGngTCloM/NVE8w7DzhvKu1LkjTovDKUJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFTfX0HklTUJ8W17VfXBNG0qBwj1ZqUGa2fSxYekPb6ZIGj0ErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFbRD0wVI24tDz72ZjZtGu15+aNmN21xm7pzZ3HH20VMpS1JhBq00TTZuGmX9imN72mY3YSypWXYdS5JUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUUE/u3hMRHwQuAOZl5hMREcDHgN8Ffgqcmpm39WJd0qDaZeEyDrl8WY/bBOjtHYEk9daUgzYi9gV+B3iwZfJbgQPqx+HA/6r/K223nlm7wtvkSduhXnQdfwQ4E8iWaScAV2TlFmDXiNirB+uSJGmgTCloI+J44JHMvGPcrPnAQy2vH66nSZK0Xdlm13FE/BOwZ5tZy4EPAUe3e1ubadlmGhFxGnAawH777betciRJGijbDNrMfHO76RFxCLA/cEd17BP7ALdFxGFUe7D7tiy+D/Boh/YvBi4GGB4ebhvGkiQNqhfddZyZd2XmHpk5lJlDVOH62sx8HLgeeHdUjgA2ZuZjvSlZkqTB0ZPTe9q4ierUnnVUp/e8p9B6JEnqaz0L2nqvdux5Au/vVdvSTNHr03Hmzpnd0/Yk9V6pPVpJ40zmHNqhZTf2/JxbSc3wEoySJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkF7dB0AdL2LCI6zzt/62mZWbAaSSUYtFKDDE5p5rPrWJKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSoo+ulaqxGxAfhBgaZ3B54o0G4J1lrGINUKg1WvtZZhreWUqHdBZs5rN6OvgraUiBjJzOGm6+iGtZYxSLXCYNVrrWVYaznTXa9dx5IkFWTQSpJU0PYStBc3XcAkWGsZg1QrDFa91lqGtZYzrfVuF2O0kiQ1ZXvZo5UkqREDH7QRcWJEZEQc1DLtgIi4ISIeiIhbI2J1RPy7et6pEbEhIm5veRw8jfUuj4h7IuLOet2HR8SXI+I7LfVcWy/71xHx38e992+msdbNdT13RMRtEfFv6+lD9Tb/85Zld4+I0Yi4qH59TkR8cLpqnUTdm+p590bE30ZE4/8GOnwmTo+IdfV23r3pGsd0qPXK+vN7d0RcGhGzm64TOtZ6Sf25uDMiro2InZuuc0y7elvmXRgRzzZZX6sO23ZlRHy/5XvsNU3XCR1rjYg4LyLuj4i1EfGfixaRmQP9AK4BvgqcU7/eCbgfOL5lmUXAqfXzU4GLGqr1dcA3gB3r17sDewNfBobbLP9y4HvAvwH2B74P7DqN9T7b8vwtwFfq50PAA8C3W+a/D7h9bNsC5wAfbGg7T1T33fXzHYB/AX6viRq7+Ez8el3vemD3JmvsotbfBaJ+rALe18e1vrxlmQ8Dy5qudaJ66+fDwKdbP9f9WCuwEjip6fq6rPU9wBXAL9XT9yhZxw4MsPrX6OuBNwDXU325vxP4RmZeP7ZcZt4N3N1EjePsBTyRmT8DyMwnACKi7cKZ+ZOIWA5cVE/6H5n59HQU2sbLgR+3vN4ErI2I4cwcAd5O9aNn7yaKm8D4ugHIzOcj4uvAq6a/pC20/UwAj0Lnz0ZDJqwVICK+BezTQG3jdaoVgKg27BygXw5S6fTdMAu4AHgHcGJz5W1hUt9jDetU6/uAd2TmC/X0H5UsovFusyl6G/DFzLwfeCoiXgv8GnDbNt739nFdx3OKV1q5Gdi37q74eEQc1TLvypZ6LhibmJmrgF+m+iX+6Wmqc8ycup77gE8Bfz5u/lXAyRGxD7CZli/chm2rbiLipcCbgLumu7hxJvpM9JsJa627jN8FfLGR6rbUsdaIuAx4HDgIuLCpAsfpVO/pwPWZ+ViDtY030efgvLqL9iMRsWNTBbboVOsrqXJgJCK+EBEHlCxi0IN2MdWXPfV/F49fICKuq8eOPtcy+erMfE3LY9N0FJuZzwK/AZwGbACujohT69nvbKnnv7XUvw+wJ7B3A+NJm+p6DgKOAa6ILX+2fhH4HartfvU01zaRiep+ZUTcDnwNuDEzv9BYlWzzM9FXuqj148C/ZOZXGyhvCxPVmpnvoep5WUvVE9O4DvV+CPh9+ufHADDhtj2L6sfLbwKvAJY2VeOYCWrdEXguq6tDfRK4tHQhA/kAdqPqvvwB1TjWQ8CDwBLg8nHLDgNfrp+fSkNjtG3+hpOAf6DDGG29zGeBU4AVwAXTXN+z417/ENiDLcc6L6XaO9itddvSJ2O0neru18fYZ6Ll9Xr6ZIx2olqBs4G/px7z6rfH+O1aTzsKuKHp2iaod7T+t7W+frwArGu6ti637W/347Zt+c69DxiqpwWwseR6B3mP9iTgisxckJlDmbkv1cFC9wOvj4jjW5Z9aSMVjhMRB47rongNE9xEISLeShUQV1B1f54Y03iE9LhaDgJmAU+Om/VXwNLMHD+9L0xQd1+Y7GeiSZ1qjYg/ojrobHHWY15N61DrgxHxqnp+AP+e6gu3cR3q/URm7ll/vw0BP83Mpo8pmOhzsFc9P6iG9Ro/LmaCf19/D7yxnnYUVW4UM8gHQy2m2str9VmqgwaOAz4cER+l2pt5BviLluXeHhFHtrz+T5n59ZLF1nYGLoyIXYHngXVUXRrXUo3RjnVhP0H1N3yU6ii+BP5vRJxJdWDUG7dquYw5dTcrVL/6TsnMza29x5l5D3DPNNXTrW3W3Ufafibq0w3OpBo2uDMibsrMP2qwTuj8+X2c6svrG/U2/lxm/lljVVba1fpe4LqIeDnV5+IOqqPl+0GnbduPOtV6TUTMo9q2t1Nt76Z1qvV5qu/c/wI8CxT9t+WVoSRJKmiQu44lSep7Bq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJU0P8HygEFzo7FNDEAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
### Regularization
We now investigate regularization techniques for Linear Regression, to reduce the variance of the model. We will concentrate of ridge regularization, though other possible techniques are lasso and elastic-net regularizations.
To use Ridge regularization (which adds a penalty term which is proportional to the sum of the squares of the coefficients), we need to find the optimal value of the tuning parameter alpha. Here we will first graphically compare the training and testing scores to determine the value of alpha for which we obtain the maximum value of the testing score. To generate the figure we create an array of alpha values, which in this case are logarithmically distributed, and perform a Ridge regularization for each, and store the testing and training scores ( $R^2$ ). Then we plot these against alpha. From the figure we see the optimal value occurs at alpha approximately 20.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# create an array of 21 alpha values logarithmically distributed between 10**(-2) and 10**2
alfas = np.logspace(-2,2,num=21)
# create two arrays for storage of the same size as alfas, but filled with zeros
ridge_training_score = np.zeros_like(alfas)
ridge_rsquared = np.zeros_like(alfas)
# loop over the values in the alfas array, at each loop the current value is alfa
# and idx is incremented by 1, starting at 0
for idx, alfa in enumerate(alfas):
ridge = Ridge(alpha=alfa) # instantatiate Ridge regularization with the current alfa
ridge.fit(X_train,Y_train) # train the model to our data set
# calculate the training score and store in the array ridge_training_score
ridge_training_score[idx] = ridge.score(X_train,Y_train)
preds_linear = ridge.predict(X_test) # calculate the model prediction for the test data
# calculate the correlation between the predicted and actual test data and store in the
# array ridge_rsquared
ridge_rsquared[idx] = r2_score(Y_test,preds_linear)
# plot the training score against alpha
plt.scatter(alfas,ridge_training_score,label='Training score')
# plot the testing score against alpha
plt.scatter(alfas,ridge_rsquared,label='Testing score')
plt.xscale('log') # make the x-axis a logarithmic scale
plt.gca().set_xlim(left=.01, right=100); # fix the x-axis limits
plt.title('Ridge regularization')
plt.xlabel('alpha');
plt.legend(loc='best');
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXhU5Z3/8feXGCBqJApWloAbWl0UQoAYo6i1rk+Aj/zUKtpF21q1tmq7rWzhuqxF7G5RaqlYWsrWp1qVUhVKLRitLraiAuEpPEmhrNUEXCOVBzEowe/vj5nEIZkkM5kzmTPJ53VduTLnPve55zs34XzPuc+Z+5i7IyIiXVu3TAcgIiKZp2QgIiJKBiIiomQgIiIoGYiICEoGIiKCkoGkmZnNMrPvt7Lezey4jowpaGZWFP0ch7Rz+8+b2aY0xPUlM3s+6HalczJ9z0BSYWZvAscAB4APgOeAW9z9gwS3d+B4d9+StiDTzMyKgP8Fct29vqvGINlNZwYShIvd/XBgODACmJTheA5iEaH8W2/v2YRI0EL5H0Syk7u/A1QQSQoAmNkjZvbDmOUJZrbdzLaZ2Vdjtzez3mb2BzPbbWbLzeyHZvZKzPoTzOwFM/uHmW0ysytbisXMFpvZf5rZEuBD4LNm1svMHoy+f020/Zxo/Rwzu8/M3jOz/zWzW2KHfszsTTM7N6b9yWb2mxbe+ytmttHM9pjZVjO7KWbdWWZWbWbfM7N3gIcbyqLrrzKzD2J+PjKzxdF1F5rZqmj/vG1mk2Pe9s/R3zuj2400sy836b/Tov26K/r7tCb9dbeZLYnG/byZ9Wmpf6XzUTKQwJhZf2AMEHfIx8xGA7cD5wHHA+c2qTIT2Av0Ba6L/jRsexjwAvAE8BngauDnZjaklZDGAzcC+cDfgUeBeuA4Imcw5wNfi9a9IRr7cKAUGJvAR27Ju8BFwBHAV4DpZlYas74vcBTwz9H4Grn7b9398OiZVj9gK/BkdPVe4FqgALgQuNnMGuI8M/q7ILr9a7HtmtlRwB+BGUBv4CfAH82sd0y1a6LxfgboTuTfSroIJQMJwnwz2wO8TWRH+IMW6l0JPOzu69x9LzC5YUX0CP1y4Afu/qG7byCy825wEfCmuz/s7vXuvhJ4Griilbgecff10TH0o4js7L/t7nvd/V1gOjAuJrb73b3a3d8HpibVAzHc/Y/u/jePeBl4Hvh8TJVPop/zI3evi9dGdFjrCWCxu/8y2u5id1/r7p+4exWRJPGFBMO6ENjs7o9F++9J4A3g4pg6D7v7X6MxzSXmDE86PyUDCcJYd88HzgJOAFoaXuhHJGE0+HvM66OBQ5qsj339z8ApZraz4Qf4EpGj7JY03T4X2B6z/S+JHAXHiy32dVLMbIyZvR4dztoJXMDBfVLr7vvaaOY/iZzR3BbT7ilm9j9mVmtmu4Cv03JfN9WPg/ub6HJhzPI7Ma8/BA5PsG3pBJQMJDDRo+BHgB+3UGU7MCBm+diY17VEhnD6x5TF1n0beNndC2J+Dnf3m1sLqcn2HwF9YrY/wt0bhpm2t/LeEBmiOTRmOW4SMrMeRM5Yfgwc4+4FwELAWogrXhvjiAyDXeHu+2NWPQEsAAa4ey9gVky7bd0WuI1IQox1LFDTxnbSRSgZSNB+CpxnZvGGGOYCXzazwWZ2KDHDSe5+AHgGmGxmh5rZCUTGxxs8C/yLmY03s9zoz8lmdmIiQbn7diLDNfeZ2RFm1s3MPmdmDcMsc4FvmVmhmRUA32vSxGpgXPR9y2h5eKo70INocjOzMUSuTSTEzEYADxA526ptsjof+Ie77zOzciJj/A1qiQw/fbaFphcS6b9rzOwQM7sKGEykX0WUDCRY0R3Yr4FmXzRz90VEksVLRC4yv9Skyi1ALyLDFY8RGRP/KLrtHiI71XFEjnLfAe4hsuNN1LVEdtYbgPeBp4B/iq77byLJogpYRWTnWU/k+xNEP8/notvdReQovZlonLcRSS7vE9lhL0gixkuBI4FXYu4oWhRd9w1gSvT6zJ3R92h43w+JDC0tiQ6Dndokrh1Errt8F9gB/Adwkbu/l0Rs0onpS2cSWmZ2D9DX3a9rs3Lw7z0GmOXuTYdWRDolnRlIaES/R1BiEeXA9cC8DnrvPDO7IDqEUkhkCKtD3lskDJQMJEzyiVw32EtkCOQ+4Pcd9N5GZPjnfSLDRBuJDMWIdAkaJhIREZ0ZiIiIkoGIiBD5xmeo9OnTx4uKijIdhohIVlmxYsV77n50e7cPXTIoKiqisrIy02GIiGQVM2s63UhSNEwkIiJKBiIiomQgIiIoGYiICEoGIiKCkoGIiKBkICIiKBmIiAhKBiIiQoLJwMxGm9kmM9tiZhNbqXeFmXn0sYBEHxH4qJmtNbONZjYpqMBFRCQ4bSYDM8sBZgJjiDwz9WozGxynXj6Rx/0tjSn+ItDD3YcCJwE3mVlR6mGLiEiQEjkzKAe2uPtWd/8YmEPkOa1N3Q3cC+yLKXPgMDM7BMgDPgZ2pxayiIgELZFkUAi8HbNcHS1rZGYjgAHu/myTbZ8i8tSq7cBbwI/d/R9N38DMbjSzSjOrrK2tTSZ+EREJQCLJwOKUNT4ezcy6AdOB78apVw4cAPoBA4HvmtlnmzXmPtvdy9y97Oij2z0Dq4iItFMiU1hXAwNilvsD22KW84FiYLGZAfQFFpjZJcA1wHPuvh9418yWAGXA1gBiFxGRgCRyZrAcON7MBppZd2AcsKBhpbvvcvc+7l7k7kXA68Al7l5JZGjobIs4DDgVeCPwTyEiIilpMxm4ez1wC1ABbATmuvt6M5sSPfpvzUzgcGAdkaTysLtXpRiziIgEzNy97VodqKyszPWkMxGR5JjZCncva+/2+gayiIgoGYiIiJKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiKCkoGIiJDYFNYdam3NLk6f+hITRg1i7IjCtjeIMX9VDdMqNrFtZx39CvKyto0wxBCWNsIQQ1g+h0g65UyePDnTMRzkh/c9MNlPOJeX/1pL/yPzOOGfjkhou/mrapj0zFr+8eHHAOzZV5+VbYQhhrC0EYYYwvI5Gtq5/tFKfvjsBn5XWU3vw7ontb10bnfdddf2yZMnz27v9qEdJqrbf4BpFZsSrj+tYhN1+w9kfRthiCEsbYQhhiDaCCKGhoRSs7MOB2p21jHpmbXMX1WTcBsN7Zw+9SUGTvwjp099KentpfMK3TBRrG0761Kum21thCGGsLQRhhiCaCOIGFpLKIkONzUklIZ2GhIKoCEvCe+ZAUC/gryU62ZbG2GIISxthCGGINoIIoZ0J5REBXWGIuET2mSQl5vDhFGDEq4/YdQg8nJzsr6NMMQQljbCEEMQbQQRQ2dKKBJOoRwmKmzHqWdD3VROX8PQRhhiCEsbYYghLJ9jwqhBBw3xQPsSSk2cHX9HJxQJJz32UiRLBHF7a7yE8qPLhibczulTX4qbUAoL8lgy8eyE49A1h+Cl+tjLUJ4ZiEhzY0cUprTTDMMZSlAXsSV4SgYiXUimE0oQd0VJeigZiEhSUkkouuYQXqG9m0hEOp8g7oqS9FAyEJEOE8RttpIeGiYSkQ4TxEVsSQ8lAxHpUKlexAbdnpoOSgYiklV0e2p6JHTNwMxGm9kmM9tiZhNbqXeFmbmZlcWUlZjZa2a23szWmlnPIAIXka5JU2KkR5tnBmaWA8wEzgOqgeVmtsDdNzSplw/cBiyNKTsE+A0w3t3XmFlvYH+A8YtIF6PbU9MjkTODcmCLu29194+BOcClcerdDdwL7IspOx+ocvc1AO6+w90PxNlWRCQhuj01PRJJBoXA2zHL1dGyRmY2Ahjg7s822fZfADezCjNbaWb/Ee8NzOxGM6s0s8ra2tokwheRrka3p6ZHIheQLU5Z4+x2ZtYNmA58uYX2zwBOBj4EXoxOpvTiQY25zwZmQ2SiuoQiF5EuSbenpkciyaAaGBCz3B/YFrOcDxQDi80MoC+wwMwuiW77sru/B2BmC4FS4KBkICKSjCBuT5WDJTJMtBw43swGmll3YBywoGGlu+9y9z7uXuTuRcDrwCXuXglUACVmdmj0YvIXgA3N30JERDKpzWTg7vXALUR27BuBue6+3symRI/+W9v2feAnRBLKamClu/8x9bBFRCRIeriNiEgnkOrDbTRRnYiIKBmIiIiSgYiIoInqRKSL0synB1MyEJEuRzOfNqdhIhHpcjTzaXNKBiLS5Wjm0+aUDESky9HMp80pGYhIl6OZT5vTBWQR6XI082lzSgYi0iVp5tODaZhIRESUDERERMlARERQMhAREZQMREQEJQMREUHJQEREUDIQERGUDEREBCUDEeloVXNhejFMLoj8rpqbmTbkIJqOQkQ6TtVc+MNtsD86VfSutyPLACVXdlwb0ozODEQkOakclb845dOdeIP9dZHyjmxDmtGZgYgkLtWj8l3VyZWnqw1pRmcGIpK4VI/Ke/VPrjxdbUgzSgYiXUmqF15TPSo/507IbfI0sdy8SHmigmhDmlEyEOkqGoZ4dr0N+KdDPMkkhFSPykuuhItnQK8BgEV+XzwjuQu/QbQBuiOpCXP3tiuZjQbuB3KAX7n71BbqXQH8DjjZ3Stjyo8FNgCT3f3Hrb1XWVmZV1ZWtlZFRNpjenE0ETTRawD8+7rE2mh6zQAiR+Xt2RlnUkCfY/6qmtA8Lc3MVrh7WXu3b/PMwMxygJnAGGAwcLWZDY5TLx+4DVgap5npwKL2BikiZH6IB4I7Ks+0AO5Imr+qhknPrKVmZx0O1OysY9Iza5m/qibYWDtIIncTlQNb3H0rgJnNAS4lcqQf627gXuD22EIzGwtsBfamHK1IVxXEvfW9+rdwZpDkhdeSK7Nv599UAIlxWsUm6vYfOKisbv8BplVsysrHaSZyzaAQiP0Lqo6WNTKzEcAAd3+2SflhwPeAu1p7AzO70cwqzayytrY2ocBFskqqR/VB3FuvC6+fCuCOpG0765IqD7tEkoHFKWu80GBm3YgMA303Tr27gOnu/kFrb+Dus929zN3Ljj766ARCEskiQVy41RBPsAJIjP0K8pIqD7tEhomqgQExy/2BbTHL+UAxsNjMAPoCC8zsEuAU4AozuxcoAD4xs33u/rMgghfpMFVzI0fhu6ojR4/n3Jn4TrS1o3oN8WRGQx+0998UmDBqEJOeWXvQUFFebg4TRg0KOtoOkUgyWA4cb2YDgRpgHHBNw0p33wX0aVg2s8XA7dG7iT4fUz4Z+ECJQDpcKjvyhu0z/a3bc+6Mf/dLVxziCUqKibHhukBY7iZKVZvJwN3rzewWoILIraUPuft6M5sCVLr7gnQHKV1YpnfkkPqRfRBH9QEcyUrwxo4ozNqdf1MJfc+gI+l7BgFJdScahjaCuBc8iHvrJxcQc5kshsHknW1v31nuzZdQS/v3DDrcttWZneM8DG0EsX2qFyzD0EYQd9AEMUQThm/diqRZ+JJBNu+8gmgjiBjCMk1wqm2EYUcOwdySWXJl5Exk8s7IbyUCCZkQJoOobNx5BdFGWI6Gw9BGmHbkOrKXTi7czzPItp1XEG0EdTSc6gXLMLQRxB00QV141S2Z0smF98wAOn6O8zC0EZaj4TC0EdQRuYZoRNoU3mSQjTuvINoIy7BGmNrQjlwk7cJ3a2m/Q7xywgnZeStkUG0EEYOIdCmp3loavmSg7xmISLYI0YFbqskg3BeQRUTCKohvt4dIeK8ZiIiEWRC3gYeIkoGISHsEcRt4iCgZiIi0RxC3gYeIkoGISHt0sifHKRmIiLRHJ5umRHcTiYi0VyeapkRnBiIiomQgIiJKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiKCkoGIiJBgMjCz0Wa2ycy2mNnEVupdYWZuZmXR5fPMbIWZrY3+PjuowEVEJDhtzlpqZjnATOA8oBpYbmYL3H1Dk3r5wG3A0pji94CL3X2bmRUDFUBhUMGLiEgwEjkzKAe2uPtWd/8YmANcGqfe3cC9wL6GAndf5e7boovrgZ5m1iPFmEVEJGCJJINC4O2Y5WqaHN2b2QhggLs/20o7lwOr3P2jpivM7EYzqzSzytra2gRCEhGRICXycBuLU+aNK826AdOBL7fYgNkQ4B7g/Hjr3X02MBugrKzM49UREels5q+qYVrFJrbtrKNfQR4TRg1i7IjMjKQnkgyqgQExy/2BbTHL+UAxsNjMAPoCC8zsEnevNLP+wDzgWnf/WzBhi4hkt/mrapj0zFrq9h8AoGZnHZOeWQuQkYSQyDDRcuB4MxtoZt2BccCChpXuvsvd+7h7kbsXAa8DDYmgAPgjMMndl6QhfhGRrDStYlNjImhQt/8A0yo2ZSSeNpOBu9cDtxC5E2gjMNfd15vZFDO7pI3NbwGOA75vZqujP59JOWoRkSy3bWddUuXplsgwEe6+EFjYpOzOFuqeFfP6h8APU4hPRKRT6leQR02cHX+/grwMRKNvIIuIZMSEUYPIy805qCwvN4cJowZlJJ6EzgxERCRYDReJs+luIhERSYOxIwoztvNvSsNEIiKiZCAiIkoGIiKZUzUXphfD5ILI76q5GQtF1wxERDKhai784TbYH729dNfbkWWAkis7PBydGYiIZMKLUz5NBA3210XKM0DJQEQkE3ZVJ1eeZkoGIiKZ0Kt/cuVppmQgIpIJ59wJuU2mnsjNi5RngJKBiEgmlFwJF8+AXgMAi/y+eEZGLh6D7iYSEcmckisztvNvSmcGIiKiZCAiIkoGIiKCkoGIiKBkICIiKBmIiAhKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICAkmAzMbbWabzGyLmU1spd4VZuZmVhZTNim63SYzGxVE0CIiEqw2n2dgZjnATOA8oBpYbmYL3H1Dk3r5wG3A0piywcA4YAjQD/iTmf2Lux8I7iOIiEiqEjkzKAe2uPtWd/8YmANcGqfe3cC9wL6YskuBOe7+kbv/L7Al2p6IiIRIIsmgEHg7Zrk6WtbIzEYAA9z92WS3jW5/o5lVmlllbW1tQoGLiEhwEkkGFqfMG1eadQOmA99NdtvGAvfZ7l7m7mVHH310AiGJiEiQEnkGcjUwIGa5P7AtZjkfKAYWmxlAX2CBmV2SwLYiIhICiZwZLAeON7OBZtadyAXhBQ0r3X2Xu/dx9yJ3LwJeBy5x98povXFm1sPMBgLHA8sC/xQiIpKSNs8M3L3ezG4BKoAc4CF3X29mU4BKd1/QyrbrzWwusAGoB76pO4lERMLH3JsN4WdUWVmZV1ZWZjoMEZGsYmYr3L2s7Zrx6RvIIiKiZCAiIkoGIiKCkoGIiKBkICIiKBmIiAhKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiJCYo+9zLj9+/dTXV3Nvn37Mh2KtKFnz57079+f3NzcTIci0iXMX1XDtIpNdO973EmptJMVyaC6upr8/HyKioqIPmdZQsjd2bFjB9XV1QwcODDT4Yh0evNX1TDpmbXU7U/9AZJZMUy0b98+evfurUQQcmZG7969dQYn0kGmVWwKJBFAliQDQIkgS+jfSaTjbNtZF1hbWZMMRETkYP0K8gJrS8kgATt27GD48OEMHz6cvn37UlhY2Lj88ccfJ9TGV77yFTZt2tRqnZkzZ/L4448HEbKIdAETRg0iLzcnkLay4gJyshqurm/bWUe/gjwmjBrE2BGF7W6vd+/erF69GoDJkydz+OGHc/vttx9Ux91xd7p1i59fH3744Tbf55vf/Ga7Y0yntj6biGRGw35tWsUmtqfYVqf7391wdb1mZx0O1OysY9Iza5m/qibw99qyZQvFxcV8/etfp7S0lO3bt3PjjTdSVlbGkCFDmDJlSmPdM844g9WrV1NfX09BQQETJ05k2LBhjBw5knfffReAO+64g5/+9KeN9SdOnEh5eTmDBg3i1VdfBWDv3r1cfvnlDBs2jKuvvpqysrLGRBVrwoQJDB48mJKSEr73ve8B8M4773DppZdSUlLCsGHDWLp0KQD33nsvxcXFFBcX88ADD7T42RYtWsTIkSMpLS3lqquuYu/evYH3qYgkZ2zOEpb0uI2htjWldjpdMoh3db1u/wGmVbQ+RNNeGzZs4Prrr2fVqlUUFhYydepUKisrWbNmDS+88AIbNmxots2uXbv4whe+wJo1axg5ciQPPfRQ3LbdnWXLljFt2rTGxPLAAw/Qt29f1qxZw8SJE1m1alWz7f7v//6PhQsXsn79eqqqqpg0aRIQOfM477zzqKqqYsWKFZx44oksW7aMxx9/nGXLlvHaa6/x85//nKqqqmafLTc3l6lTp/Liiy+ycuVKSkpKuP/++4PqRhFpj6q58IfbYNfbKTfV6ZJBS1fXg7zqHutzn/scJ598cuPyk08+SWlpKaWlpWzcuDFuMsjLy2PMmDEAnHTSSbz55ptx277sssua1XnllVcYN24cAMOGDWPIkCHNtjvqqKPo1q0bN9xwA/PmzeOwww4DYPHixdx0000AHHLIIRxxxBH85S9/4fLLL+fQQw8lPz+fsWPH8sorrzT7bK+++iobNmzgtNNOY/jw4Tz++OMtxi0iHeTFKbA/mH1bp7tm0K8gj5o4O/4gr7rHatjRAmzevJn777+fZcuWUVBQwL/927/Fvee+e/fuja9zcnKor6+P23aPHj2a1XH3NmPKzc2lsrKSF154gTlz5vCLX/yC559/Hmh+62dr7cV+Nndn9OjRPPbYY22+v4h0kF3VgTWV0JmBmY02s01mtsXMJsZZ/3UzW2tmq83sFTMbHC3PNbNHo+s2mtmkwCJvQbyr63m5OUwYNSjdb83u3bvJz8/niCOOYPv27VRUVAT+HmeccQZz584FYO3atXHPPPbs2cPu3bu56KKLmD59euNQ0r/+678ya9YsAA4cOMDu3bs588wzmTdvHnV1dXzwwQf8/ve/5/Of/3yzNk877TRefvlltm6NjEvu3buXzZs3B/75RCQJvfoH1lSbycDMcoCZwBhgMHB1w84+xhPuPtTdhwP3Aj+Jln8R6OHuQ4GTgJvMrCig2OMaO6KQH102lMKCPAwoLMjjR5cNTeluokSVlpYyePBgiouLueGGGzj99NMDf49bb72VmpoaSkpKuO+++yguLqZXr14H1dm1axcXXnghw4YN4+yzz+YnP4n8c/zsZz+joqKCoUOHUlZWxhtvvEF5eTlXX301J598Mqeeeio333wzQ4cObfa+xxxzDA8++CBXXXUVw4YN47TTTuOvf/1r4J9PRJJwzp2QG8yoh7U17GBmI4HJ7j4qujwJwN1/1EL9q4Fr3X1M9PU1wP8DegGvAae6+z9aer+ysjKvrKw8qGzjxo2ceOKJCX+ozqy+vp76+np69uzJ5s2bOf/889m8eTOHHBKeET/9e4l0oKq58OIUyqZtpHLbgXZPAZDIHqQQiL1UXQ2c0rSSmX0T+A7QHTg7WvwUcCmwHTgU+PfWEoG07YMPPuCcc86hvr4ed+eXv/xlqBKBiHSwkiuh5EpWfMdWpNJMInuReJmm2emEu88EZprZNcAdwHVAOXAA6AccCfzFzP7k7gfdEGtmNwI3Ahx77LFJfYCupqCggBUrUvo3FxFpJpELyNXAgJjl/sC2VurPAcZGX18DPOfu+939XWAJUNZ0A3ef7e5l7l529NFHJxa5iIgEJpFksBw43swGmll3YBywILaCmR0fs3gh0HCbyVvA2RZxGHAq8EbqYYuISJDaHCZy93ozuwWoAHKAh9x9vZlNASrdfQFwi5mdC+wH3icyRASRu5AeBtYRGW562N2r0vA5REQkBQldeXT3hcDCJmV3xrz+VgvbfUDk9lIREQmxTjcdRToEMYU1wEMPPcQ777zTuJzItNYiIh2hc96TGL3vll3VkW/onXNn5PardkpkCutEPPTQQ5SWltK3b18gsWmtM6G+vl63q4p0MZ3vzOCgWfw88vsPt0XK0+DRRx+lvLyc4cOH841vfINPPvmE+vp6xo8fz9ChQykuLmbGjBn89re/ZfXq1Vx11VWNZxSJTGu9efNmTjnlFMrLy/n+979PQUFBsxj27NnDmDFjGDZsGMXFxTz11FMALF26lJEjRzJs2DBOOeUUPvzwQ+rq6rjuuusYOnQopaWl/PnPfwbgV7/6FePGjeOiiy5qnERv6tSplJeXU1JSctB03CLS+XS+ZBBvFr/9dZHygK1bt4558+bx6quvNu7U58yZw4oVK3jvvfdYu3Yt69at49prr21MAg1JIXayOmh5Wutbb72V22+/nWXLlnHMMcfEjWPhwoUUFRWxZs0a1q1bx3nnnce+ffsYN24cM2fOZM2aNTz//PP06NGDGTNm0L17d9auXctjjz3G+PHjG4e6XnvtNR577DFeeOEFFi5cyFtvvcXSpUtZvXo1r776auMzFUSk8+l8yaClWfwCnN2vwZ/+9CeWL19OWVkZw4cP5+WXX+Zvf/sbxx13HJs2beJb3/oWFRUVzeYOiqelaa2XLl3K5ZdfDsA111wTd9uSkhKee+45Jk6cyJIlS+jVqxcbN27k2GOPpbS0FIBevXqRk5PDK6+8wvjx4wEYMmQI/fr1Y8uWLQCcf/75HHnkkQA8//zzLFq0iBEjRlBaWsqWLVs0F5FIJ9b5BoZ79Y//oIcAZ/dr4O589atf5e677262rqqqikWLFjFjxgyefvppZs+e3WpbiU5rHc+JJ55IZWUlCxcuZMKECVx00UWMHj262XTVDTG3pOmU1XfccQfXX399wnGISPbqfGcG8Wbxy82LlAfs3HPPZe7cubz33ntA5K6jt956i9raWtydL37xi9x1112sXLkSgPz8fPbs2ZPUe5SXlzNv3jwA5syZE7dOTU0Nhx9+OOPHj+c73/kOK1euZMiQIfz9739vfO/du3dz4MABzjzzTB5//HEgMqHc9u3bOe6445q1OWrUKB588MHGR1tWV1c3fk4R6Xw635lBw11DAd5N1JKhQ4fygx/8gHPPPZdPPvmE3NxcZs2aRU5ODtdffz3ujs5quGMAAAXLSURBVJlxzz33AJFbSb/2ta+Rl5fHsmXLEnqPGTNmMH78eO655x4uuOCCuENODY/A7NatG927d2fWrFn06NGDJ598kptvvpl9+/aRl5fHSy+9xK233spNN93E0KFDyc3N5de//nWz6xcAF1xwAW+88QannnoqEElkTzzxBH369Emhx0QkrNqcwrqjaQrrg+3du5dDDz0UM+M3v/kN8+bN4+mnn850WK3qyv9eIpliZivcvdncb4nqfGcGnczy5cv59re/zSeffMKRRx4Z2u8miEh2UzIIubPOOqvxC28iIumSNReQwzacJfHp30kkO2VFMujZsyc7duzQjibk3J0dO3bQs2fPTIciIknKimGi/v37U11dTW1tbaZDkTb07NmT/v2D/06HiKRXViSD3NxcBg4cmOkwREQ6rawYJhIRkfRSMhARESUDEREJ4TeQzWwP0BGP/+oF7Erzdm3VbWl9MuVNy5ou9wHSPalQe/symW0Tqaf+DK4/W1ufSL8lUtYRfdlSHEFvl4n+bLo8yN3z2w61Be4eqh+gsoPeZ3a6t2urbkvrkylvWhZnOe392d6+TGbbROqpP4Prz9bWJ9JviZTp/3pq/Rn032ZXHib6Qwds11bdltYnU960rL2fKxWpvGei2yZST/0ZXH+2tj6RfkumLN3C/H+9pXUd/rcZxmGiSk9hsiU5mPozWOrP4Kgvg5Vqf4bxzKD1p8BIstSfwVJ/Bkd9GayU+jN0ZwYiItLxwnhmICIiHUzJQERElAxERCTLkoGZjTWz/zaz35vZ+ZmOJ9uZ2WfN7EEzeyrTsWQjMzvMzB6N/k1+KdPxZDv9PQYr2f1lhyUDM3vIzN41s3VNykeb2SYz22JmE1trw93nu/sNwJeBq9IYbugF1J9b3f369EaaXZLs18uAp6J/k5d0eLBZIJn+1N9j25Lsz6T2lx15ZvAIMDq2wMxygJnAGGAwcLWZDTazoWb2bJOfz8Rsekd0u67sEYLrT/nUIyTYr0B/4O1otQMdGGM2eYTE+1Pa9gjJ92dC+8sOe56Bu//ZzIqaFJcDW9x9K4CZzQEudfcfARc1bcPMDJgKLHL3lemNONyC6E9pLpl+BaqJJITVZNmQa0dJsj83dGx02SeZ/jSzjSSxv8z0H3Ahnx5ZQeQ/V2Er9W8FzgWuMLOvpzOwLJVUf5pZbzObBYwws0npDi6LtdSvzwCXm9kvyMw0C9kqbn/q77HdWvr7TGp/meknnVmcsha/BefuM4AZ6Qsn6yXbnzsAJdW2xe1Xd98LfKWjg+kEWupP/T22T0v9mdT+MtNnBtXAgJjl/sC2DMXSGag/00P9Giz1Z7AC6c9MJ4PlwPFmNtDMugPjgAUZjimbqT/TQ/0aLPVnsALpz468tfRJ4DVgkJlVm9n17l4P3AJUABuBue6+vqNiymbqz/RQvwZL/RmsdPanJqoTEZGMDxOJiEgIKBmIiIiSgYiIKBmIiAhKBiIigpKBiIigZCByEDN708z6pFpHJNsoGYiIiJKBdF1mNt/MVpjZejO7scm6IjN7wyJPMqsys6fM7NCYKrea2UozW2tmJ0S3KTezV81sVfT3oA79QCIpUDKQruyr7n4SUAbcZma9m6wfBMx29xJgN/CNmHXvuXsp8Avg9mjZG8CZ7j4CuBP4r7RGLxIgJQPpym4zszXA60RmfTy+yfq33X1J9PVvgDNi1j0T/b0CKIq+7gX8LvpIwunAkHQELZIOSgbSJZnZWUQe/DHS3YcBq4CeTao1nbgrdvmj6O8DfPpckLuB/3H3YuDiOO2JhJaSgXRVvYD33f3D6Jj/qXHqHGtmI6OvrwZeSaDNmujrLwcSpUgHUTKQruo54BAzqyJyRP96nDobgeuidY4icn2gNfcCPzKzJUBOkMGKpJumsBaJI/rQ8WejQz4inZ7ODERERGcGIiKiMwMREUHJQEREUDIQERGUDEREBCUDERFByUBERID/DxutjlcTT+qCAAAAAElFTkSuQmCC)
%% Cell type:markdown id: tags:
Now can investigate Ridge regularization using the optimal value of alpha $\approx$ 20. Now we see significantly reduced variance in the coefficients and that the most important variables are `BMI`, `BP` and `S5`. Note that `S1`-`S4` have varied significantly, due to the collinearity (i.e., the variables are highly correlated).
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
feature_names = nX.columns.tolist() # write the column names to a list
# cross_validate takes the particular model, in this case ridge regularization
# and undertakes a number of runs according the method specified by cv=
# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times,
# giving 25 runs
# return_estimator=True returns the fitting data for each run
scores = cross_validate(Ridge(alpha=20.), nX, Y,
cv=RepeatedKFold(n_splits=5, n_repeats=5), return_estimator=True)
# take the results for each simulation (estimator), extract the coefficients for each run
# and add them to a dataframe with columns being the feature names
coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)
# plot the descriptive statics of the coefficients in a box and whisker plot to show
# variability
ax = coefs.plot(kind='box',figsize=(10,5))
plt.title('Ridge regression, alpha = 20.')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
print('Testing score is',
round(np.mean(scores['test_score']),4),"±",round(np.std(scores['test_score']),4))
```
%%%% Output: stream
Testing score is 0.489 ± 0.0538
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAeZklEQVR4nO3dfbRcd13v8feHEmuwj9BCnxvlQULLatHw4CVLCEItoAIuEAILWwwU0CLei9JA1BZprq3IwxVQKKa2uGiAxYPUFkt5KMUooim00HKUxwCFlIaCoYXITXu/94/ZwcnJnJOTnvmdmTl5v9aadWb23rP39+wzZz7z++3f7J2qQpIktXGPURcgSdJiZtBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQauJkeQtSf5olvmV5AELWdO42tu+WqAaHpvk5mEvK00ag1ZjI8mWJDuS3JHkliSXJDlo1/yqelFVvXqUNU4K99W+S/LkJJuS/Gf3+ntbkoP75h+Y5OIk3+/m/69R1qvJYdBq3PxqVR0EnAo8DHjFiOvZTXqG9n8z7PVpXg4FzgeOAZYDxwGv6Zt/HvBA4ERgFfDyJKcvcI2aQP6DayxV1S3Ah+gFLgBdC/f8vsd/kGRrkm8l+a3+5ye5T5K/71of/5bk/CSb+uY/OMmHk3w3yX8k+Y2Zakny8STrk/wT8EPgZ5IcmmRDt/1vdus/oFv+gCSvTfKdJF9NcnbXrX3Pu7m+ByS5Nsn2bp3v6qYnyeuT3NrN+2ySk2fYVy9I8qXu9708yTF98yrJi5J8Mcn3krw5Sebyd0ryvCRTSW5P8pUkL5xl2S1JXpHk8912/ibJT05b5mXd77M1yfP6pj85yWe6v+c3kpw3l/r2RVVdVlVXVdUPq+p7wNuAR/ct8pvAq6vqe1U11c0/c9h1aPExaDWWkhwHPBH40gzzTwd+H3gCvVbG46ct8mbgB8BRwBndbddzfwr4MHAZcF9gNfCXSU6apaTnAmcBBwNfAy4F7gQeQK/lfRrw/G7ZF3S1nwr8HPDUea7v1cDVwOH0Wllv7KafBvwi8CDgMOCZwG3TN5TkccCfAr8BHN1t753TFvsV4OHAKd1yvzzLvuh3a/fcQ4DnAa9P8nOzLP+cbt337+r+w755R9FrVR4LrAHenOTwbt4P6AXdYcCTgRcnGbRfSXJC1/070+3Zc/zdfhG4qVvn4fRaujf0zb8BmO01I/VUlTdvY3EDtgB3ALcDBXwUOKxv/iXA+d39i4EL+uY9qHvOA4ADgJ3Az/bNPx/Y1N1/JvCP07b9VuDcGer6OPAnfY/vB/wIWNo3bTVwTXf/Y8AL++Y9vqvtnndzfW8HLgKOm1bX44AvAI8C7jFtXv++2gD8Wd+8g7r9s6x7XMDKvvnvBtbezb/h3wEv7e4/Frh52t/3RX2PnwR8uW/ZHbv2UTftVuBRM2znDcDrG74WnwB8D3hQ9/j4bj/95LRltoz6/8bb+N9s0WrcPLWqDqb3xvtg4IgZljsG+Ebf46/13T8SuOe0+f33TwQe2d/KodfSOmqWuqY/fwmwte/5b6XXOh5UW//9u7O+lwMB/jXJTbu6yavqY8Cb6LXev53koiSHDNjWMfTtn6q6g17L99i+ZW7pu/9DemG8V0memORfui7p/6QXnjP9zWDPv9kxfY9vq6o7B9WR5JFJrkmyLcl24EV72c7dluRR9Ho7nl5VX+gm39H97N+/h9D7UCjNyqDVWKqqa+m1yv58hkW20mtl7HJC3/1t9Lphj+ub1r/sN4Brq+qwvttBVfXi2Uqa9vwfAUf0Pf+QqtrVjbh1lm3v8/qq6paqekFVHQO8kF439wO6eX9RVT9PrwvzQcAfDNjWt+iFOfDjrvP7AN+c5ffdqyQHAu+l9ze6X1UdBnyQ3oeCmUz/m31rjpu7DLgcOL6qDgXeMtN2uq7jO2a5PWeW3+lh3XZ+q6o+umt69Y7ZbqXXtb7LKXRdy9JsDFqNszcAT0hy6oB57wbOTPKQJPcCzt01o6ruAt4HnJfkXkkeTO/43i5XAA9K8twkS7rbw5Msn0tRVbWV3jHT1yY5JMk9ktw/yWP6antpkmOTHAacM5/1JXlGd8waet2ZBdzV1fzIJEvoHcP8L+CuAZu4DHheklO7cPzfwKeqasveftcky7rBUssGzP4J4EC6DzZJnkjvuPFsfifJcUnuDbwSeNfeaugcDHy3qv4rySOAGY+zVtXXuw9OM93eMeh53UCyq4CXVNXfD1jk7cAfJjm8e029gN6HQWlWBq3GVlVto/fmtseJF6rqH+gF8cfoDZj62LRFzqY3sOYW4G+BjfRajVTV7fQC4Vn0WlS3ABfSC425+k16QfN5euH3HnoDjaA3GvVq4LPAZ+i18u5kcAjOZX0PBz6V5A56ra2XVtVX6XVdvq1b/mv0uoP36AHoWmZ/RK/1uZXeQKRnzfH3PL5b9x6t324//i69Dxbfoxd+l+9lfZfR2zdf6W7nz774j/028CdJbgf+uNvmsL2M3mGHDX2t3/4W67nAl+ntj2uB11TVVbBbK/qEPdaq/V6qvPC7Fr8kFwJHVdUZe114+Nt+IvCWqjpxrwuPmSR/CGyrqrcOYV1bgOdX1UfmXZg0Qe456gKkFrquvZ8APkevRbiG//66TOttL6V3QoOr6Y0oPhd4/0Jse9iqaq4tTkkzsOtYi9XB9I7T/oBeN+NrgQ8s0LYDvIped+pngCl63Z2S9kN2HUuS1JAtWkmSGjJoJUlqaKwGQx1xxBG1bNmyUZchSdI+ue66675TVUcOmjdWQbts2TI2b9486jIkSdonSb420zy7jiVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpobE6BaPGV5J9Wt7LL0pSjy1azUlV7XE78ZwrBk43ZCXpvxm0kiQ1ZNBKktSQQStJUkPzDtokxye5JslUkpuSvLSbfl6Sbya5vrs9af7lSpI0WYYx6vhO4GVV9ekkBwPXJflwN+/1VfXnQ9iGJEkTad5BW1Vbga3d/duTTAHHzne9kiQtBkM9RptkGfAw4FPdpLOTfDbJxUkOH+a2JEkLL8k+3TTEoE1yEPBe4Peq6vvAXwH3B06l1+J97QzPOyvJ5iSbt23bNqxyJEkN+J36fTeUM0MlWUIvZN9RVe8DqKpv981/G3DFoOdW1UXARQArVqzwrzJip7zqarbv2Dnn5ZetvXJOyx26dAk3nHva3S1LkibWvIM2vb6BDcBUVb2ub/rR3fFbgKcBN853W2pv+46dbLngyUNf71wDWZIWm2G0aB8NPBf4XJLru2mvBFYnORUoYAvwwiFsS5KkiTKMUcebgEFHvD8433VLkjTpvHqPdnPw8rU89NK1DdYLMPwuaUkadwatdnP71AUeo5UE7NvgSAdGzsyglSQN1GJw5P74oduLCkiS1JAtWu2hxSfOQ5cuGfo6JWkSGLTazb50Ey1be2WT47mStJjYdSxJUkMGrSRJDRm0kiQ1ZNBKktSQg6EkSQO1OFPc/niWOINWkjRQizPFecIKSZI0VAatJEkNGbSSJDVk0EqS1JCDoTQnSQZPv3Dw8lXVsBpJmhy2aDUnVfXj22WXXcZJJ53EPe5xD0466SQuu+yy3eYbspL032zRap9s3LiRdevWsWHDBlauXMmmTZtYs2YNAKtXrx5xdZI0fmzRap+sX7+eDRs2sGrVKpYsWcKqVavYsGED69evH3VpkjSWbNFqn0xNTbFy5crdpq1cuZKpqakRVSSppWGfYGJ/vDa1Qat9snz5cjZt2sSqVat+PG3Tpk0sX758hFVJamGuZ4Xy2tSzm3fXcZLjk1yTZCrJTUle2k2/d5IPJ/li9/Pw+ZerUVu3bh1r1qzhmmuuYefOnVxzzTWsWbOGdevWjbo0SRpLw2jR3gm8rKo+neRg4LokHwbOBD5aVRckWQusBc4ZwvY0QrsGPL3kJS9hamqK5cuXs379egdCSdIM5h20VbUV2Nrdvz3JFHAs8BTgsd1ilwIfx6BdFFavXm2wStIcDXXUcZJlwMOATwH360J4Vxjfd5jbkiRpEgwtaJMcBLwX+L2q+v4+PO+sJJuTbN62bduwypEkaSwMZdRxkiX0QvYdVfW+bvK3kxxdVVuTHA3cOui5VXURcBHAihUrPKWQJI0xT8e674Yx6jjABmCqql7XN+ty4Izu/hnAB+a7LUnSaE0/3erebhpOi/bRwHOBzyW5vpv2SuAC4N1J1gBfB54xhG1JkjRRhjHqeBMwuC8Bfmm+65ckaZJ5rmNJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpoWFc+F0aK8lMl0cerKoaVSJJtmi1CFXVHrcTz7li4HRDVlJrBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ359R5NtFNedTXbd+yc07LL1l45p+UOXbqEG849bT5lSdKPDSVok1wM/Apwa1Wd3E07D3gBsK1b7JVV9cFhbE/aZfuOnWy54MlDXedcA1mS5mJYXceXAKcPmP76qjq1uxmykqT9zlCCtqo+AXx3GOuSJGkxaT0Y6uwkn01ycZLDG29LkqSx0zJo/wq4P3AqsBV47aCFkpyVZHOSzdu2bRu0iCRJEyvDOtdrkmXAFbsGQ811Xr8VK1bU5s2bh1KP9g8PvfShTdb7uTM+12S9khanJNdV1YpB85p9vSfJ0VW1tXv4NODGVtvS/uv2qQscdSxprA3r6z0bgccCRyS5GTgXeGySU4ECtgAvHMa2JEmaJEMJ2qpaPWDyhmGsW5KkSeYpGCVJashTMGriDfuY6qFLlwx1fZL2bwatJtpcB0ItW3vl0AdNSdJc2HUsSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDnhlKi06SwdMvHLz8sK7JLEmDGLRadAxOSePErmNJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGhhK0SS5OcmuSG/um3TvJh5N8sft5+DC2JUnSJBlWi/YS4PRp09YCH62qBwIf7R5LkrRfGUrQVtUngO9Om/wU4NLu/qXAU4exLUmSJknLY7T3q6qtAN3P+zbcliRJY2nkg6GSnJVkc5LN27ZtG3U5kiQNVcug/XaSowG6n7cOWqiqLqqqFVW14sgjj2xYjiRJC69l0F4OnNHdPwP4QMNtSZI0lob19Z6NwCeBn01yc5I1wAXAE5J8EXhC91iSpP3KUC78XlWrZ5j1S8NYvyRJk2rkg6EkSVrMDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoaytd7JEl3X5J9Wr6qGlWiFmzRStKIVdUetxPPuWLgdEN28hi0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDXlmKElaIKe86mq279g55+WXrb1yTssdunQJN5x72t0tS40ZtJK0QLbv2MmWC5489PXONZA1GnYdS5LUkEErSVJDBq0kSQ0ZtJIkNeRgKEnSojQu1/lt3qJNsiXJ55Jcn2Rz6+1JkgSDr/M727V+W1moFu2qqvrOAm1LksbSwcvX8tBL1zZYL8Dwvzak4bDrWJIWyO1TF/g92v3QQgyGKuDqJNclOWsBtidJ0thYiBbto6vqW0nuC3w4yb9X1Sd2zezC9yyAE044YQHKkSRp4TRv0VbVt7qftwLvBx4xbf5FVbWiqlYceeSRrcuRJGlBNQ3aJD+V5OBd94HTgBtbblOSpHHSuuv4fsD7u+8y3RO4rKquarxNSZLGRtOgraqvAKe03IYkSS0uQTisyw/69R5JWkAtvopz6NIlQ1/npGlxCcJh/a0MWklaIPsSBMvWXtnkO7eLVYuTgQzrRCAGrSRp4rU4GciwWrRevUeSpIYMWkmSGjJoJUlqyGO0kqRFYdgjuoc1mtuglSRNvHEe0W3XsSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDTnqWJJGrLuU6J7TLxy8fFU1rGbxmGm/wuB922q/GrSSNGIGZxvjsl/tOpYkqaFF1aKdrZtgkHH5tCNJWrwWVYu2qgbeTjznioHTJUlqbVEFrSRJ48aglSSpIYNWkqSGJnYw1CmvuprtO3bOefm5Xj7p0KVLuOHc0+5uWZIk7aZ50CY5Hfg/wAHAX1fVBcNY7/YdO5tc5mjY1zOUJO3fmnYdJzkAeDPwROAhwOokD2m5TUmSxknrY7SPAL5UVV+pqv8LvBN4SuNtSpI0Nlp3HR8LfKPv8c3AI2da+LbbbuOSSy6Z04pP/4nbuOSSbfMqbiHXK026a6+9dp+Wf8xjHtOoEmmytA7aQadq2u1MEUnOAs4COPbYYxuXI+numik4/+Urt/Gon7nPAlcjTY60PENSkl8AzquqX+4evwKgqv500PIrVqyozZs3z2ndy9Ze2WwwVIv1SouV/zMSJLmuqlYMmte6RftvwAOT/DTwTeBZwLOHseKDl6/loZeuHcaqpq0XwDcNSdJwNA3aqrozydnAh+h9vefiqrppGOu+feoCv94jSRp7zb9HW1UfBD7YejuSJI2jiT0zlKR2PPOaNDwTHbQtunkPXbpk6OuUJo1nXpOGZ2KDdl/eBBwVKUkaFa/eI0lSQxPbopUWg2TQOV1m1vJ775LasEUrjVBVDbydeM4VA6dLmjwGrSRJDRm0kiQ1ZNBKktTQohoMNdvAkly45zSPeUmSWltUQWtwSsPhRTuk4VlUQStpOLxohzQ8HqOVJKkhg1aSpIbsOpY0kBftkIbDoJW0By/aIQ2PXceSJDVk0EqS1JBBK0lSQwatJEkNGbSSJDVk0Epa9DZu3MjJJ5/MAQccwMknn8zGjRtHXZL2I369R9KitnHjRtatW8eGDRtYuXIlmzZtYs2aNQCsXr16xNVpf9CsRZvkvCTfTHJ9d3tSq21J0kzWr1/Phg0bWLVqFUuWLGHVqlVs2LCB9evXj7o07Sdat2hfX1V/3ngbkjSjqakpVq5cudu0lStXMjU1NaKKtL+x61haIKe86mq279g55+XncgrEQ5cu4YZzT5tPWYve8uXL2bRpE6tWrfrxtE2bNrF8+fIRVqX9SeugPTvJbwKbgZdV1fcab08aW9t37Bz6qQq97NzerVu3jjVr1uxxjNauYy2UeQVtko8ARw2YtQ74K+DVQHU/Xwv81oB1nAWcBXDCCSfMpxxJ2sOuAU8veclLmJqaYvny5axfv96BUFow8wraqnr8XJZL8jbgihnWcRFwEcCKFStqPvVI0iCrV682WDUyLUcdH9338GnAja22JUnSuGp5jPbPkpxKr+t4C/DChtuSJGksNQvaqnpuq3VLkjQp/HqPpDlJMvO8C/ecVuWQCwkMWklzZHBKd49BKy2Qg5ev5aGXrh3yOgGG+91cScNl0EoL5PapCzxhhbQf8jJ5kiQ1ZNBKktSQQStJUkMeo5W0KM32daTpHFGtlgxaaQENe/DSoUuXDHV9i8mg8Fy29sqhD0iT9saglRbIvrzBGwjS4uExWkmSGrJFK2minfKqq9m+Y+ecl59r9/2hS5dww7mn3d2ypB8zaEfIwRrS/G3fsbNJN7snA9Gw2HU8QlW1x+3Ec64YOF2SNJkMWkmSGrLrWNJEa3Gxht56wQs2aBgMWkkTrcXFGsBjtBoeu44lSWrIoJUkqSGDVpKkhgxaSZIacjCUpInXYuCSF2zQsMwraJM8AzgPWA48oqo29817BbAGuAv43ar60Hy2JUmDeLEGjbv5tmhvBH4deGv/xCQPAZ4FnAQcA3wkyYOq6q55bk9aVGY7DWcu3HOaZwmTJs+8graqpmDgm8VTgHdW1Y+Aryb5EvAI4JPz2d6k8qTnmonBKS1+rY7RHgv8S9/jm7tp+yVPei5J+6+9Bm2SjwBHDZi1rqo+MNPTBkwb+NE9yVnAWQAnnHDC3sqRJGmi7DVoq+rxd2O9NwPH9z0+DvjWDOu/CLgIYMWKFfajSZIWlVbfo70ceFaSA5P8NPBA4F8bbUuSpLE1r6BN8rQkNwO/AFyZ5EMAVXUT8G7g88BVwO844liStD+a76jj9wPvn2HeemD9fNa/WHgZL0naf3lmqAXgZbwkaf/luY4lSWrIFu0C8Vys0sKa6axbnnFLC82gXQCei1VaeIanxoVdx5IkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNeSZoUbIU8RJ0uJn0I6Q4SlJi59dx5IkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDGafz7SbZBnytwaqPAL7TYL0tWGsbk1QrTFa91tqGtbbTot4Tq+rIQTPGKmhbSbK5qlaMuo65sNY2JqlWmKx6rbUNa21noeu161iSpIYMWkmSGtpfgvaiURewD6y1jUmqFSarXmttw1rbWdB694tjtJIkjcr+0qKVJGkkJj5okzwtSSV5cN+0Bya5IsmXk1yX5Jokv9jNOzPJtiTX990esoD1rktyU5LPdtt+ZJKPJ/mPvnre0y37F0n+aNpz37yAtd7V1XNDkk8n+R/d9GXdPn9137JHJNmZ5E3d4/OS/P5C1boPde/o5n0+yVuSjPx/YIbXxNlJvtTt5yNGXeMuM9T6ju71e2OSi5MsGXWdMGOtG7rXxWeTvCfJQaOuc5dB9fbNe2OSO0ZZX78Z9u0lSb7a9z526qjrhBlrTZL1Sb6QZCrJ7zYtoqom+ga8G/hH4Lzu8U8CXwB+rW+Zk4Ezu/tnAm8aUa2/AHwSOLB7fARwDPBxYMWA5Q8BvgL8DPDTwFeBwxaw3jv67v8ycG13fxnwZeAzffNfDFy/a98C5wG/P6L9PFvdN3b37wl8Avj1UdQ4h9fEw7p6twBHjLLGOdT6JCDdbSPw4jGu9ZC+ZV4HrB11rbPV291fAfxt/+t6HGsFLgGePur65ljr84C3A/fopt+3ZR33ZIJ1n0YfDawCLqf35v4c4JNVdfmu5arqRuDGUdQ4zdHAd6rqRwBV9R2AJAMXrqrvJ1kHvKmb9MdV9Z8LUegAhwDf63u8A5hKsqKqNgPPpPeh55hRFDeL6XUDUFV3Jvln4AELX9JuBr4mgG/BzK+NEZm1VoAk/wocN4LappupVgDS27FLgXEZpDLTe8MBwGuAZwNPG115u9mn97ERm6nWFwPPrqr/102/tWURI+82m6enAldV1ReA7yb5OeAk4NN7ed4zp3UdL21eac/VwPFdd8VfJnlM37x39NXzml0Tq2ojcDi9T+J/u0B17rK0q+ffgb8GXj1t/juBZyU5DriLvjfcEdtb3SS5F/BLwOcWurhpZntNjJtZa+26jJ8LXDWS6nY3Y61J/ga4BXgw8MZRFTjNTPWeDVxeVVtHWNt0s70O1nddtK9PcuCoCuwzU633p5cDm5P8Q5IHtixi0oN2Nb03e7qfq6cvkOT93bGj9/VNfldVndp327EQxVbVHcDPA2cB24B3JTmzm/2cvnr+oK/+44CjgGNGcDxpR1fPg4HTgbdn94+tVwFPoLff37XAtc1mtrrvn+R64J+AK6vqH0ZWJXt9TYyVOdT6l8AnquofR1DebmartaqeR6/nZYpeT8zIzVDvK4FnMD4fBoBZ9+0r6H14eThwb+CcUdW4yyy1Hgj8V/XODvU24OLWhUzkDbgPve7Lr9E7jvUN4OvAGuDSacuuAD7e3T+TER2jHfA7PB34e2Y4Rtst817gDOAC4DULXN8d0x5/G7gvux/rvJhe6+A+/fuWMTlGO1Pd43rb9Zroe7yFMTlGO1utwLnA39Ed8xq32/T92k17DHDFqGubpd6d3f/Wlu72/4Avjbq2Oe7bx47jvu17z/13YFk3LcD2ltud5Bbt04G3V9WJVbWsqo6nN1joC8Cjk/xa37L3GkmF0yT52WldFKcyy0UUkjyRXkC8nV7359OygCOkp9XyYOAA4LZps14LnFNV06ePhVnqHgv7+poYpZlqTfJ8eoPOVld3zGvUZqj160ke0M0P8Kv03nBHboZ631pVR3Xvb8uAH1bVqMcUzPY6OLqbH3qH9UY+LmaW/6+/Ax7XTXsMvdxoZpIHQ62m18rr9156gwZ+BXhdkjfQa83cDpzft9wzk6zse/zbVfXPLYvtHAS8MclhwJ3Al+h1abyH3jHaXV3Y36H3O7yB3ii+An6Q5OX0BkY9bo81t7G062aF3qe+M6rqrv7e46q6CbhpgeqZq73WPUYGvia6rxu8nN5hg88m+WBVPX+EdcLMr99b6L15fbLbx++rqj8ZWZU9g2p9EfD+JIfQe13cQG+0/DiYad+Oo5lqfXeSI+nt2+vp7e9Rm6nWO+m95/5P4A6g6f+WZ4aSJKmhSe46liRp7Bm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkP/HyZcyzdwfQPxAAAAAElFTkSuQmCC)
%% Cell type:markdown id: tags:
### Exercise
%% Cell type:markdown id: tags:
Use `RidgeCV` to estimate the optimal value of `alpha` using the array defined above. You should also investigate the
effect of changing the number of cross-validations in the range [4,10]. In general, you should get a similar optimal value of `alpha` to the example above. The use of cross-validations, where the testing and training sets are changed, creates a more robust method than the graphical method above. (4 marks)
%% Cell type:code id: tags:
``` python
from sklearn.linear_model import RidgeCV
```
%% Cell type:markdown id: tags:
There are two possible ways to approach this. The first is to use the training sets to train the model, and the testing sets to test the model. In this case you want to calculate the score using `ridge.score`. This gives an optimal `alpha` of approximately 25.
%% Cell type:code id: tags:
``` python
X_train, X_test, Y_train, Y_test = train_test_split(nX, Y, test_size=0.8)
for CV in range(3,11):
ridge = RidgeCV(alphas=alfas,cv=CV).fit(X_train, Y_train)
scores = ridge.score(X_test, Y_test)
alfa_opt = ridge.alpha_
print('cv=',CV,'Testing score =',round(scores,4),'Optimal alpha = ',round(alfa_opt,2))
```
%%%% Output: stream
cv= 3 Testing score = 0.4626 Optimal alpha = 25.12
cv= 4 Testing score = 0.4629 Optimal alpha = 15.85
cv= 5 Testing score = 0.4626 Optimal alpha = 25.12
cv= 6 Testing score = 0.4626 Optimal alpha = 25.12
cv= 7 Testing score = 0.4626 Optimal alpha = 25.12
cv= 8 Testing score = 0.4574 Optimal alpha = 39.81
cv= 9 Testing score = 0.4626 Optimal alpha = 25.12
cv= 10 Testing score = 0.4203 Optimal alpha = 100.0
%% Cell type:markdown id: tags:
The second way is to use `nX` and `Y`, and the cross validation routine will return the best score and the best `alpha` as `ridge.best_score_` and `ridge.alpha_`, respectively. This gives an optimal `alpha` of approximately 40.
If you use `cv=RepeatedKFold(n_splits=CV, n_repeats=5)`, then results are consistent with method above, but take longer to run.
%% Cell type:code id: tags:
``` python
for CV in range(3,11):
ridge = RidgeCV(alphas=alfas,cv=CV).fit(X_train, Y_train)
scores = ridge.best_score_
alfa_opt = ridge.alpha_
print('cv=',CV,'Testing score =',round(scores,4),'Optimal alpha = ',round(alfa_opt,2))
```
%%%% Output: stream
cv= 3 Testing score = 0.4313 Optimal alpha = 25.12
cv= 4 Testing score = 0.4957 Optimal alpha = 15.85
cv= 5 Testing score = 0.3883 Optimal alpha = 25.12