Unverified Commit 3e49c1fb authored by Simon Bowly's avatar Simon Bowly
Browse files

Regularization draft

parent b11b975b
%% Cell type:markdown id: tags:
# Regularization in Linear Regression
In this lesson we will investigate the effect of various features on the tendency of patients to develop diabetes, using the `sklearn` methods `LinearRegression` and `Ridge` and `Lasso` regularization. We will look at how coefficients can vary signficantly due to overfitting, and how that can be alleviated using regularization.
First import the Diabetes dataset. The same dataset can be imported from the `sklearn` example datasets, but is already normalized. We will use the unnormalized dataset initially, to show why data needs to be normalised before modelling.
%% Cell type:code id: tags:
``` python
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from sklearn.model_selection import train_test_split # for splitting the data into training and testing sets
from sklearn.linear_model import LinearRegression, Lasso, Ridge, LassoCV, RidgeCV # models we are going to use
from sklearn.metrics import r2_score # for comparing the predicted and test values
import seaborn as sns
```
%% Cell type:markdown id: tags:
## Recall
Read data, check correlations, construct normalised regression model, as we did in the [multi-linear regression notebook](https://gitlab.erc.monash.edu.au/bads/data-challenges-resources/-/blob/main/Machine-Learning/Supervised-Methods/Regression/Multivariate-Linear-Regression.ipynb).
%% Cell type:code id: tags:
``` python
df = pd.read_csv('Diabetes_Data.csv') # read the Diabetes dataset in to a pandas dataframe
corrs = df.corr() # calculate the correlation table
# as this is a symmetric table, set up a mask so that we only plot values below the main diagonal
mask = np.triu(np.ones_like(corrs, dtype=bool))
f, ax = plt.subplots(figsize=(10, 8)) # initialise the plots and axes
# plot the correlations as a seaborn heatmap, with a colourbar
sns.heatmap(corrs, mask=mask, center=0, annot=True, square=True, linewidths=.5)
# do some fiddling so that the top and bottom are not obscured
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5);
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAf4AAAHSCAYAAAANAaloAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACF00lEQVR4nOzdd3gUxR/H8ffcJZCEHtLovffee5EqIipgQ0URpIl0EFQURVFQUZEiiuWnIIj03qsU6Z1Q0y6dmoTkbn5/JFxyBBLgLkXu+3qee7jdnb393N6R2ZmdvVVaa4QQQgjhHAxZHUAIIYQQmUcqfiGEEMKJSMUvhBBCOBGp+IUQQggnIhW/EEII4USk4hdCCCGciEsmbEOuFxRCCPGwVKZu7fhix9dVVXpk7nt4QNLiF0IIIZyIVPxCCCGEE8mMrn4hhBAiW9Nms8NfM1v28yMtfiGEEMKpSItfCCGEMCdkdYJMIy1+IYQQwolIi18IIYTT0xbHt/iz6zl+qfiFEEKIDBjcl11JV78QQgjhRKTFL4QQwulpGdwnhBBCiMeRtPiFEEIIafELIYQQ4nEkLX4hhBBOLyMu58uupOIXQggh5HI+IYQQQjyOpMUvhBDC6cnlfEIIIYR4LEmLXwghhJAWvxBCCCEeR1LxCyGEEE5EKn4hhBBOT1vMDn88CKVUB6XUaaXUOaXUmHssL66U2qyUOqiUOqKU6mTve5WKXwghhMgCSikj8C3QEagM9FZKVb6r2LvAQq11LaAX8J2925XBfUIIIZxeFl3OVx84p7U+D6CU+gPoBpxIGQ3Im/Q8HxBk70al4hdCCCGypuIvAlxJMR0ANLirzPvAOqXUYCAX0NbejUpXvxBCCJEBlFL9lFL7Uzz6PcLL9AZ+0loXBToBvyil7Kq7pcUvhBDC6T3oYLyHek2tZwOz0ygSCBRLMV00aV5KfYEOSa+3WynlBngBoY+a675HDUqpvGksK/6oGxRCCCEEAPuAckqpUkqpHCQO3lt2V5nLQBsApVQlwA0Is2ejaXUXbLnzRCm18a5lf9uzUSGEECJbMSc4/pEOrXUCMAhYC5wkcfT+caXUJKXUk0nFhgNvKKUOA78Dr2ittT1vNa2ufpXiuWcay4QQQoj/tKy6SY/WehWw6q55E1M8PwE0ceQ202rx6/s8v9e0EEIIIf4D0mrx+yil3iGxdX/nOUnT3hmeTAghhMgsTnSTnrQq/jlAnns8B5ibYYmEEEIIkWHuW/FrrT/IzCBCCCFEVsmIy/myq/tW/EqpKkAZrfWypOnpJP5cIMA3Wut/MyGfEEIIIRworcF9U4DwFNNPACuBzcDEe66RJOWvFc2endZvFwghhBDZQBZczpdV0jrHX0hrvSvF9DWt9WIApdSbab3oXb9WJFcACCGEyNa02Xm6+tNq8acczIfWumGKSZ+MiSOEEEKIjJRWiz9IKdVAa/1PyplKqYY44LaAQgghRHaRVT/gkxXSqvhHAwuUUj8Bdwby1QH6AD0zOJcQQgghMsB9u/q11nuBhoAReIXECl8BLyc9F0IIIcR/TJq35dVam4CJSqnaJN4TuA/QHFicCdmEEEKIzGGRrn6UUuVJrOx7k3hZ3wJAaa1bZVI2IYQQQjhYWi3+U8B2oIvW+hyAUmpYpqQSQgghMpFczpfoaSAY2KyUmqOUaoPcjlcIIYT4T0vrt/r/Bv5WSuUCugFvk3iXvpnAEq31ukxJKIQQQmQ0J2rxpzm4D0BrfRP4H/A/pVQB4FkSL/WTil8IIcRjwZmu40+rqz8VrXWU1nq21rpNRgUSQgghRMZJt8UvhBBCPPacqKv/oVr8QgghhPhvkxa/EEIIpyeX8wkhhBDisSQtfiGEEE5PW5ynxS8VvxBCCOFEXf2ZUvF3LFctMzbzQFafPZrVEYQQQogsIy1+IYQQTk8G9wkhhBDisSQtfiGEEE5Pmy1ZHSHTSMUvhBBCOFHFL139QgghhBORil8IIYRwIlLxCyGEEE5EzvELIYRwes50OZ9U/EIIIZyeNuusjpBp0uzqV0rNUErlucf8ikqpDRkXSwghhBAZIb1z/CHAIaXU8wBKKQ+l1GfAcuDbjA4nhBBCZAZttjj8kV2lWfFrrScD7YAXlFLbgCNAAlBDa70kE/IJIYQQwoEeZFT/ncMWF8AInNRa33JkiDrNmjBn7TJ+2LCSZ/v1TbW8ar06zPh7AStOHqRph3bW+dUb1OObZX9aH0uP7adR29aOjCaEEMIJSIs/iVJqArAB+Flr3RhoCnRTSm1VSlV2SACDgYHvj2fC62/xZsdutOzSkeJlS9uUCQ0K5ovRE9i8fJXN/CP/7GPQk88y6MlnGfNSX+JiYvl3xy5HxBJCCCEeS+mN6vcCammtrwNorQOBZ5RSHYHFQCV7A5SvXo2gS5cJuRIAwNaVq2nYphWXz523lgkNDCJp+/d9nWYd2rN/2w7iYmPtjSSEEMLJaIuM6gdAaz30TqV/1/zVQE1HBPDy8yEsOMQ6HR5ioqCv70O/TvPOHdiyYlX6BYUQQoi7aLN2+CO7Sq+rf2GK55/etXh5Guv1U0rtV0rtnz17tp0R01fA24tSFcpxYLt08wshhBBpSW9wX7kUz9vdtcz7fitprWdrretqrev269cvzQ2Eh4TiXcjPOu3l50uEyZROLFvNOz3BrnWbMCckPNR6QgghBIA2O/6RXaVX8afVV+GQfowzR49RuGQJfIsWwcXVhRadO7Jn45aHeo2WXTpKN78QQgjxANIb3OehlKpF4gGCu1KqdtJ8Bbg7IoDFbGbmBx/z0bzvMRqNrFu0hMvn/Hlp6EDOHD3OP5u2UL5aFSZ89xW58+ahQasWvDjkLfp36g6AT5HCePn5cXTvfkfEEUII4YSy6py8UqoD8BWJl8vP1VpPuUeZ54D3SWxwH9ZaP2/XNtMaKa+U2py0IZU0K2VhpbVu9QDb0B3LVXv0hA62+uzRrI4ghBAifSr9Io4T0Leuw2v+oj/sT/M9KKWMwBkST6UHAPuA3lrrEynKlAMWAq211lFKKR+tdag9udJr8Y8Grmitg5MC9AF6ABdJPPoQQgghxKOpD5zTWp8HUEr9AXQDTqQo8wbwrdY6CsDeSh/SP8f/PRCXFKg58AkwH7gKZPxwfSGEECITZNHgviLAlRTTAUnzUioPlFdK7VRK7Uk6NWCX9Fr8Rq11ZNLznsBsrfViYLFS6pC9GxdCCCEeV0qpfkDKS9tma60fttHsQuIVdi2BosA2pVQ1rXX0o+ZKt+JXSrlorROANti+gfTWFUIIIf4TMuLyu6RKPq2KPhAolmK6aNK8lAKAf7TW8cAFpdQZEg8E9j1qrvS6+n8HtiqllgIxwHYApVRZErv7hRBCCPFo9gHllFKllFI5gF7AsrvK/E1iax+llBeJXf/nsUOarXat9WSl1EagELBOJ18CYAAG27NhIYQQwplprROUUoOAtSRezjdPa31cKTUJ2K+1Xpa0rL1S6gRgBkZqrSPs2W663fVa6z33mHfGno0KIYQQ2Ykli+6iq7VeBay6a97EFM818E7SwyHS6+oXQgghxGNEBugJIYRwetn5t/UdTVr8QgghhBORFr8QQginZ7Fk6i8EZymp+IUQQji9rBrclxWkq18IIYRwItLiF0II4fScaXBfplT8citcIYQQInvIlIr/yfI1M2MzD2TZmUPULFkmq2PYOHTRP6sjCCGEU5PBfUIIIYQTsThRV78M7hNCCCGciLT4hRBCOD1n6uqXFr8QQgjhRKTFL4QQwulpJ2rxS8UvhBDC6ckv9wkhhBDisSQtfiGEEE5PBvcJIYQQ4rGUZotfKfVOWsu11tMcG0cIIYQQGSm9rv48mZJCCCGEEJkizYpfa/1BZoSo3awxr48fhdFoYN2fS1g8+0eb5VXq1ub18SMpWaEcU4eNYdfaDdZlfUYMpW7LZgAs+G42O1ate6QMjVs0Z9TECRiMRpYsWMCPM2fZLHfNkYOPpn1OpapVuRodxehBQwgKCMTFxYX3Pv2EilWqYHQxsuKvJcz77nsAXuz7Kt17PofWcPb0ad4bOYrbcbcfKZ8QQoiM40zn+NPr6v86reVa6yH2BjAYDLz53lgmvtqfiBATXyz+jb0bt3LF/7y1TFhwCF+NmchTfV+2Wbduy2aUqVKJod164prDlY9//YEDW3cSc/PmQ2cYO+l9+r/YB1NICL8tW8LW9Rs5f+6ctUz3557l2tWrPNmyNU907cLQMaMZPWgI7Tp1xDVHDp7t0Ak3Nzf+2rCWNcuWkxCfQO9X+vB02yeIi4vjs2++pkPXrixbtNiu/SWEEMLxzE5U8ac3uK8/0BQIAvYDB+562K1c9aoEX7qC6UogCfEJbF+5lgZtW9qUCQ0M4uLps2iLtplfrExpju87gMVsJi4mlounzlC7eZOHzlC1Zg2uXLpE4JUrJMTHs3b5Clq2b2tTpmX7tixf/BcAG1atpn7jRgBowN3dHaPRSE43N+Jvx3Pj+g0AjEYXcrq5YTQacXN3J8xkeuhsQgghhCOlV/EXAmYDTwAvAa7AUq31fK31fEcEKOjrQ3hIiHU6PMREQV+fB1r3wqkz1G7WhBxubuQpkJ9qDevhXcj3oTP4+PoSEhRsnTYFh+Dj63tXGT9rGbPZzI3r18lfoAAbVq0mJiaG9Xt3s2bXdn6eM5drV68SajLx85y5rNm1nfV7d3Pj+nV2b9/x0NmEEEJkPItFOfyRXaVZ8WutI7TW32utWwGvAvmBE0qplzIjXHoO7dzN/q07+GzBfEZOm8Kpg0ewmDP355eq1qiBxWymfYPGdGrWkpde70uRYsXIkzcvLdu1pXOzlrRv0Bh3Dw86PdUtU7MJIYQQd3ug6/iVUrWBocCLwGrS6eZXSvVTSu1XSu2fPXt2mq8dYQrFy8/POu3l50uEKfRBYgHw5/dzebtbTya+2h+lFIEXLz3wuneEmkz4FS5knfYt5EfoXd3yoaYQaxmj0UjuPHmIjoqiY7eu7Ny6jYSEBKIiIjh04ABVqlejYdMmBF65QlRkJAkJCWxcs5aadWo/dDYhhBAZz6KVwx/ZVZoVv1JqklLqAPAOsBWoq7Xuq7U+kdZ6WuvZWuu6Wuu6/fr1SzPA2aPHKVyyOL5FC+Pi6kKzzk/wz8atDxbeYCBP/nwAlKxQjpIVynFwx+4HWjel44ePULxkSQoXLYqLqytPdO3C1vUbbcpsXb+Rrj2eBqBtp47s25W4neCgIOv5fjd3d6rVqskFf3+Cg4KoXqsmbm5uADRo0pjz5/wfOpsQQoiMZ7E4/pFdpXcd/7vABaBG0uNjpRSAArTWurq9ASxmM7MmTeH9H2ZiMBrYsGgpV8758/yQAZw7doK9m7ZStloVxn07jdx581KvVXOeHzKAQZ17YHRx4ZP/zQMg5sZNpo0cj8VsfugMZrOZKRM/YObPP2EwGli6cBH+Z88yYNjbnDh6lK0bNrJk4UImT/uCZVs2cS06mtGDhwKw4OdfmTT1UxavWw1KsezPxZw9dRqADavX8PvKZZgTzJw6fpzFv/9h7+4SQggh7KK01vdfqFSJtFbWWj9Iv7p+snzNh4yVcZadOUTNkmWyOoaNQxelJ0AIIe6SqX3lWxs3v39l+Iha7NqWLfv70/sBn1QVu1LKC4jQaR0xCCGEECJbSu8cf0Ol1Bal1F9KqVpKqWPAMcCklOqQORGFEEKIjOVMl/Old47/G2AckA/YBHTUWu9RSlUEfgfWZHA+IYQQIsOZs/EofEdL73I+F631Oq31n0CI1noPgNb6VMZHE0IIIYSjpdfiT3lBQsxdy+QcvxBCiMdCdr7u3tHSq/hrKKWukTi60j3pOUnTbhmaTAghhBAOl96ofmNmBRFCCCGyipzjF0IIIcRjSSp+IYQQwomkd45fCCGEeOyZnWi4urT4hRBCCCciLX4hhBBOz5ku55MWvxBCCOFEpMUvhBDC6TnT5XyZUvEvO3MoMzbzwOQ2uEIIIVJypsF9mVLxT63RNjM280BGHt5A38oNszqGjR9O7OGXmu2yOobVS4fWZ3UEIYRwCkl3uv0KMAJztdZT7lOuB7AIqKe13m/PNqWrXwghhNMzk/ld/UopI/At0A4IAPYppZZprU/cVS4PMBT4xxHblcF9QgghRNaoD5zTWp/XWt8G/gC63aPch8CnQKwjNioVvxBCCKdn1o5/PIAiwJUU0wFJ86yUUrWBYlrrlY56r9LVL4QQwumZM+A1lVL9gH4pZs3WWs9+iPUNwDTgFUfmkopfCCGEyABJlXxaFX0gUCzFdNGkeXfkAaoCW5RSAH7AMqXUk/YM8JOKXwghhNPLiBb/A9gHlFNKlSKxwu8FPH9nodb6KuB1Z1optQUYYe+ofjnHL4QQQmQBrXUCMAhYC5wEFmqtjyulJimlnsyo7UqLXwghhNPLisv5ALTWq4BVd82beJ+yLR2xTWnxCyGEEE5EWvxCCCGcnlk7z2/2SsUvhBDC6WXR4L4skS26+ks2rkffpT/y+vL51H+tV6rlNZ7twiuL5tBnwff0/ulLCpYubrM8j58PQ3cvp97LzzokT9WmDZm8cgEfr/mTjq+/lGp5+z69+XD577y/5FdGzJtBwcJ+NsvdcnkwddMynh8/3CF57la4cV2e/Hse3Zb9RJVXe963XPE2TXnp0Ho8K5fPkBxCCCH+e9Kt+JVSTymlRiilnsiIAMpgoN24wSx6axzzuvelUodWqSr2k6s28dMzbzC/Z3/2/riAViMG2CxvNaI/F3bsdVieF94dwfQ3hzGha28adGpPoTIlbcpcOnmaD599hfe7v8j+tZt5Zvggm+Xdh7zJmf0HHZLnXvnqjx3MpoHjWP7065Ts0Ip8d+0vABcPdyo+352wIyczJIcQQoj/pjQrfqXUd8AwoCDwoVJqgqMDFKpagagrQVwNDMaSkMCpNVso27KJTZnbN29Zn7u6u6FTnIsp26oxVwNDCPe/5JA8patVJvRyAOEBQZjjE9i7ej21Wje3KXN677/cjo0D4PyRYxTw9bEuK1G5AnkLenJil2MORO5WsGoFrl8J4kZgCJaEBC6t3UKxlo1Tlas58BWO/7QA8+3bGZJDCCHEf1N6Lf7mQGut9VigJfCUowPk9vHiekiodfp6aBi5fQumKler55O8seJnWgx7g42ffgskHgQ0eLUXu77/2WF58vt6E5kiT1RIKPl9vO9bvunTXTm2fTcASimeGzWUhVO/dlieu3n4eHEzJMw6fdMUjruPl00Zz4pl8fD1JnB7xhx8CCHE48acAY/sKr2K/7bW2gygtb4FWXShI3BwwTLmdHmZbV/OpdEbLwDQZMDL7P91MfExDrlh0UNr2LUDJatWYs28XwFo1bsHR7ftIsoUls6aGUgp6ozoz4Fps7IugxBC/Mc4U8Wf3qj+ikqpI0nPFVAmaVoBWmtd/V4rpbwxwaxZaVdAN0LDyeOX3FWex8ebG6aI+5Y/uWYz7cYPZTVQqFolyrdtTou33yBnntxobSHh9m0O/rE0nbd1f9GmMDxT5Cng50N0aOqKvFKjenTu9wqf9RlAQnw8AGVqVqNcnRq06t2DnB7uuLi6EncrhsXTv3vkPHe7FRpOLr/kHohcvl7EhIZbp11zuZO/TEnaz/0cAPeCnrT6chKb355I5IkzDsshhBDivym9ir/So7zoXTcm0FO/XXjfssHHT1OgeBHyFfHjuimcih1asmLsxzZl8hcvQvTlxPsWlGnegKjLAQD8/uowa5nG/V8m/laMXZU+wIVjJ/EtUQyvIoWICg2jfsd2zB5l+yNKxSuV5+X3RjP9zWFcj4yyzp8z6j3r8yZPdaZElYoOrfQBIo6fJk/xIuQu7Met0HBKPNGSHeM+sS6Pv3GLP1s9Y51uN/dzDkybLZW+EEKkwYxcxw+A1jrViDmllBcQobVjfu1Amy1s+GQGz8ycgsFg4Ojfa4jwv0STt/oQcvwM/lt3U7tXN0o0rI0lPoHY6zdYNeEzR2z6nixmM79N/pxhc77CYDCwY8kKgs5doNugN7h4/BSHN2/n2RGDyenhwYDpkwGIDDIxY9DIDMuUkjZb2DvlG9rM/ARlMHBu6Vqu+l+ixoA+RJw4Q8DW3ZmSQwghxH+TSqv+Vko1BKYAkcCHwC8k3inIALystV7zANvQU2u0dUBUxxh5eAN9KzfM6hg2fjixh19qtsvqGFYvHVqf1RGEECJTx5RNrdHW4U3+kYc3ZNm4uLSk19X/DTAOyAdsAjpqrfcopSoCvwMPUvELIYQQ2Zoz/WRveqP6XbTW67TWfwIhWus9AFrrUxkfTQghhBCOll6L35Liecxdy5zn8EgIIcRjLTtffudo6VX8NZRS10g81+Ke9JykabcMTSaEEEIIh0tvVL8xs4IIIYQQWcWZLufLFnfnE0IIIUTmSK+rXwghhHjsOVOLXyp+IYQQTs+ZBvdJV78QQgjhRKTFL4QQwunJD/gIIYQQ4rEkLX4hhBBOz5kG90mLXwghhHAiUvELIYQQTiTN2/I6iPP0nwghhHCUTL2l7fBqzR1eV31xdNt/8ra8DjGwapPM2MwD+fbYTsqXKJnVMWycuXSRN6o0yuoYVnOO76Zv5YZZHcPqhxN7sjqCEEI8NmRwnxBCCKdncaLL+aTiF0II4fRkVL8QQgghHkvS4hdCCOH0pMUvhBBCiMeStPiFEEI4PWf6rX6p+IUQQjg96eoXQgghxGNJWvxCCCGcnjNdxy8tfiGEEMKJSItfCCGE03Omc/zZouKv3KQBz4x5G4PRwM7Fy1n/w682y1u/3JPGPbpiMZu5ERnNrxM+JjLYBEABP19emDSGAn4+aK35bsAIIoNCHjpDsxYtGP/eRIxGI3/+sYDZM2faLHfNkYOp06ZRpVpVoqOieXvQIAIDAqheowYffvIJAEopZnz5JevXrgVg044d3Lx5A4vZQoI5gR5dn3yU3UOVpg3pNeZtDEYj2xcvY83cX2yWt+vTi6Y9nsSSYOZ6VDQ/vTuZyODEfTDryA4Cz/oDEBFs4ttBox4pw92qNm1I77HDUEYD2xctY/Vdmdr36U2zZ57EnGDmRlQUP747mYgUn4tbLg8+XP4HBzdu5X+Tv3BIJiGEEOnL8opfGQw89+5wZrzxNtEhoYxaMJejm3cQcv6itcyVk2f5tGdf4mPjaNbzKZ4aPpB5IyYC8PIn77J29s+c2r2PnO7uWLTloTMYDAbe+3ASr77wIiEhISxetoyNG9bjf/actcyzPZ/j6tWrtGvRks5duzJyzBjeHjSIM6dP83TXrpjNZrx9vFm2ejWbNmzAbDYn5uvVm6ioKLv2z/PjhzP9jaFEmUIZv2AehzdvJ9g/ef9cPnmGyc+9yu3YOFr07M4zwwcye8QEAG7HxTGpR59H3v79Mr3w7gi+eH0IUaZQJiz4kUN3Zbp08jRbnn2F27FxtOz5NM8MH8Ss4e9al3cf8iZn9h90aC4hhHhUztTif+Rz/Eqp2Y4IULJaJcIuBxAREIQ5IYEDqzdSvXUzmzJn9/1LfGwcABcOHye/rzcAfqVLYjQaObV7HwBxMTHWcg+jes2aXLp4iStXrhAfH8/K5ctp2669TZk27dqzZPFiANasWkWjJo0BiI2NtVbyOXPmxNG3OS5VrTJhVwIIDwjCHJ/AvlUbqNmquU2Z03v/5XbS+z5/+DgF/HwcmuFupatVJvRycqa9q9dTq3UamY4co4BvcqYSlSuQt6AnJ3btzdCcQgjxoCxaO/yRXaVZ8SulPO/zKAh0ckSA/D7eRIWEWqejTaHk9/G+b/nGT3flxPbE27T6lCzGres3eOPLjxnz5490Hz4QZXj4YxlfP19CgoOs0yHBwfj6+aYqExyUWMZsNnP9+nUKFCgAJB44rFy/juVr1/Le+HetBwIazbxff+GvFcvp2bv3Q+cCyO/rTWRw8v6JMoVaD3zupWmPrhzbvts67ZojB+MXzGPs/+ZQ867K+VHl9/UmMsVnFhWS9mfW9OnkTEopnhs1lIVTv3ZIFiGEEA8nva7+MOASoFLM00nTGdusvId6XdpTvEpFvnxlIABGo5GytWvwybOvEhVs4rXPJ9HwqU7s/mtFpuY6cugQndu1p0zZMnz6xRds3bKF23FxPN/jGUwmE54FC/LTr7/i7+/P/r0Z18pt0OUJSlapyNQ+b1nnjWn3NNGhYXgVLczwed8QeNafsCuBGZbhbg27dqBk1Up89vIAAFr17sHRbbuIMoVlWgYhhEiPM3X1p1fxnwfaaK0v371AKXXlfisppfoB/QBmzZqV5gaiQ8Nsuqbz+/oQHZq6UqjQsC4d+vVh+isDSYiPByDKFEbAqbNEBCS2xI9s2kbJ6lXYnWrttJlCTPgVKmyd9itUCFOIKVWZQoULYwoJwWg0kidPnlTn7v3P+XPz1i3Kly/PsaNHMZkSXyMyIoL1a9dSvWaNh674o01heBZK3j8FfH2IvkelWalhPTr3e4Wpr7xl3T+AdV+GBwRxZt+/FKtU3u6KP9oUhmeKz6yA370/s0qNEjN91meANVOZmtUoV6cGrXr3IKeHOy6ursTdimHx9O/syiSEEP9FSqkOwFeAEZirtZ5y1/J3gNeBBBIb469prS/Zs830+sW/BArcZ9ln91tJaz1ba11Xa123X79+aW7g0rFT+BQvSsEihTC6uFCnYxuObt5hU6ZoxXL0fm8U3w8azY3I6BTrnsQ9b25yF8gPQPn6dQhJMcDsQR09fJiSpUpStFhRXF1d6dy1KxvXr7cps2nDerr36AFAh06d2L1rV2K2YkUxGo0AFC5ShNJlyhAYEIC7uzu5cuUCwN3dnSbNm3H29JmHznbx2El8ihfDq0ghjK4u1OvUlsObt9uUKVaxPC++N4pvBo3kemTywYhH3jy4uLoCkDt/PsrUqk6w/4WHznC3C8dO4lsiOVP9ju04dFem4pXK8/J7o5lxV6Y5o95jVJunGN2uO39OncGupauk0hdCOCWllBH4FugIVAZ6K6Uq31XsIFBXa10dWEQade+DSq/FvxewNn2VUi8DPUjs/n/f3o0DWMxmFn48nYGzpmEwGtm9ZAXB/hfoPPB1Lh8/xdEtO+g+fCA5Pdx5fdpHAEQGm5g1eDTaYmHJ598y5IevAMWVE6fZuWjZQ2cwm81MmjiRH37+GaPRyKKFCzl39ixD3hnGsSNH2bRhA38uWMjU6dNYv3ULV6OjGTZoMAB16taj31sDSIhPwKItfPDuBKKioihWrBjfzk4c/2h0MbJ86VK2b936SPvnf5O/4O3ZX6IMBnYuWUGQ/wWeHPQGl46f5PDmHTwzYhBuHh70nz4ZSL5sr1Dpkrz43mi0tqCUgTVzf7EZef+oLGYzv03+nGFzvsJgMLBjyQqCzl2g26A3uHj8FIc3b+fZEYPJ6eHBgKRMkUEmZgwaafe2hRAiI2TRTXrqA+e01ucBlFJ/AN2AE3cKaK03pyi/B3jR3o2qtEahK6X+BdpqrSOVUs2BP4DBQE2gktb6mQfYhh5YtYm9OR3m22M7KV+iZFbHsHHm0kXeqNIoq2NYzTm+m76VG2Z1DKsfTuzJ6ghCiMyn0i/iOM9WrOPwmv/PUwfSfA9KqWeADlrr15OmXwIaaK0H3af8N0CI1voje3Kl1+I3aq0jk573BGZrrRcDi5VSh+zZsBBCCJFdWDJgcF/K8W5JZmutH+lSeKXUi0BdoIW9udKt+JVSLlrrBKANtm8gy3/8RwghhMiukir5tCr6QKBYiumiSfNsKKXaAuOBFlrrh/+xmrukV3n/DmxVSoUDMcD2pBBlgav2blwIIYTIDrLoHP8+oJxSqhSJFX4v4PmUBZRStYBZJJ4SCE39Eg8vzYpfaz1ZKbURKASs08kDAgwknusXQggh/vOy4pf2tNYJSqlBwFoSL+ebp7U+rpSaBOzXWi8DpgK5gT+VUgCXtdaPduOXJOl212utU42s0lo//HVpQgghhLChtV4FrLpr3sQUz9s6eptynl4IIYTTc6Zf7nvkm/QIIYQQ4r9HWvxCCCGc3qPc0v2/Slr8QgghhBORFr8QQginlxE/4JNdScUvhBDC6WXRdfxZQrr6hRBCCCciLX4hhBBOz5m6+qXFL4QQQjiRNG/L6yDOcxglhBDCUTL1trwty1RyeF21xf9kpr6HB5UpXf29KtXNjM08kD9O7qd5mUpZHcPGNv+TDKzaJKtjWH17bCdPlq+Z1TGslp05BMcXZ3WMZFV6ZHUCIYSDOc9V/NLVL4QQQjgVqfiFEEIIJyIVvxBCCOFE5HI+IYQQTs/iRD/gIxW/EEIIpyfX8QshhBDisSQtfiGEEE7Pmbr6pcUvhBBCOBFp8QshhHB6co5fCCGEEI8lafELIYRwes7U4peKXwghhNOzOE+9nz0q/hpNG9Fn3AgMBgObFv3NsrnzbZZ36vMCrZ/phtls5npkFN+/O4nwoBBKVCxP3/fG4J47Fxazhb9nzWP36vV256nfvClDJozDYDSwcsEifps11zZvvboMfncspSuW54Ohw9m6Zp3Nco/cufh5zQp2rN/Ilx98ZHeeyk0a8MyYtzEYDexcvJz1P/xqs7z1yz1p3KMrFrOZG5HR/DrhYyKDTQAU8PPlhUljKODng9aa7waMIDIoxO5MtZs15vXxozAaDaz7cwmLZ/9os7xK3dq8Pn4kJSuUY+qwMexau8G6rM+IodRt2QyABd/NZscq2/1nr23/nmHyvBVYLBaebVuPfk+3sFkeGBrFuG8XE3ntFvlzuzN16HP4eeVzaAYhhMiu0qz4lVJG4HWgKLBGa70zxbJ3tdZ212rKYOC1CaOZ3HcgESYTHy/8mQObtxHof8Fa5uLJU4x7dhG3Y+No16sHL4wYwlfvjON2bCzfjXmPkEtXKODtxceLf+Xwjt3cun7jkfMYDAaGvT+Bd/r0JSzExOwlC9mxcTOXzvlby5iCgvh41Fh6vfHaPV/j9WFDOLxv/yNnSEkZDDz37nBmvPE20SGhjFowl6ObdxBy/qK1zJWTZ/m0Z1/iY+No1vMpnho+kHkjJgLw8ifvsnb2z5zavY+c7u5YtP33oDIYDLz53lgmvtqfiBATXyz+jb0bt3LF/7y1TFhwCF+NmchTfV+2Wbduy2aUqVKJod164prDlY9//YEDW3cSc/Om3bkAzGYLk+Ys48f3XsO3YF6eGfUdretVpGwxX2uZT+ev5qmWteneqja7j/rzxW9rmTr0OYdsXwjx3+RMXf3pDe6bBbQAIoCvlVLTUix72hEBylavQsjlK4QGBGKOT2DXqnXUbW3bQjux9wC3Y+MAOHv4GJ6+iX/Egy9eJuTSFQCiwsK5FhFJXs8CduWpVKM6gZcuE3wlgIT4eDauWEXTtq1tyoQEBnH+9Bm0JXUlWr5qZQp4ebFvx85Uyx5FyWqVCLscQERAEOaEBA6s3kj11s1sypzd9y/xSfvnwuHj5Pf1BsCvdEmMRiOndu8DIC4mxlrOHuWqVyX40hVMVwJJiE9g+8q1NGjb0qZMaGAQF0+fRd/Vf1asTGmO7zuAxWwmLiaWi6fOULu5425JfORcACUKFaSYnyc5XF3o3LQ6G/eetCnjHxBKw2qlAWhYtXSq5UII8ThLr+Kvr7V+Xmv9JdAAyK2U+ksplRNQjgjg6eNDRIjJOh1pCsXT1+e+5Vv16Mah7btSzS9TrQourq6YLgfYlcfL14fQ4OSu8LAQE96+vmmskUwpxcCxo/nuk8/sypBSfh9vokJCrdPRplDy+3jft3zjp7tyYvseAHxKFuPW9Ru88eXHjPnzR7oPH4gy2H8hR0FfH8JDkvdReIiJgml8ZildOHWG2s2akMPNjTwF8lOtYT28Cz3Y/n0Qpoir+BVM7rb3LZgPU+Q1mzIVS/qxbs9xANb/c5ybMXFEXb/lsAxCiP8eC9rhj+wqvVogx50nWusErXU/4DCwCch9v5WUUv2UUvuVUvtnz57tmKRA064dKV21Est/+Nlmfn7vggz8dBIzx3+AzsJfX+r+Ym/2bN1GWIoDmcxUr0t7ilepyIYf/weA0WikbO0a/PX5N3zW63UKFi1Mw6c6ZUm2Ow7t3M3+rTv4bMF8Rk6bwqmDR7CY7T/98DBG9enEvuMXeGr4DPYev4CvZ16MBoccxwoh/qO0dvwju0pvcN9+pVQHrfWaOzO01h8opQKBmfdbSWs9G7hT4+tN0+9f+UeGhlLQL7nF5+nrQ6QpNFW5qo3q0/3N1/jg5X4kxMdb57vnysXo779iwZffce7wsXTeTvrCTaH4FPKzTnv7+RJmerCKvEqtmlSvV4enXuiNu4cHrq6uxNy6xayp09Jf+T6iQ8Mo4Jfcms7v60N0aFiqchUa1qVDvz5Mf2Wgdf9EmcIIOHWWiIAgAI5s2kbJ6lXY/chpEkWYQvHyS95HXn6+RNzjM7ufP7+fy5/fJw6YHP7FJwRevGRnomS+BfMREnHVOm2KuIqvZ17bMp55+Wb0iwDcjIlj3e7j5M3l7rAMQgiRnaXX4v+KxBY+AEqpl5VSS4HqgEP6Z/2PnsCvRDG8ixTG6OpC407tObB5m02ZkpUq8Mb745g68B2uRUZZ5xtdXRg+Yyrblq7kn3UbHRGHU0eOUrRkCQoVLYKLqyttunRi58bND7Tuh++M4tlmbejZoi3fTfmMtUuW2lXpA1w6dgqf4kUpWKQQRhcX6nRsw9HNO2zKFK1Yjt7vjeL7QaO5ERmdYt2TuOfNTe4C+QEoX78OIf4X7coDcPbocQqXLI5v0cK4uLrQrPMT/LNx6wOtazAYyJM/sSu+ZIVylKxQjoM77D0USVatbBEuBodzxRTJ7fgEVu44Qut6lWzKRF67iSVpfMbsv7bSo00dh21fCPHf5Exd/em1+GcBbQGUUs2BKcBgoCaJLfpn7A1gMZv58aOpjJs7A4PByOa/lhFw7jzPDn6T88dOcmDzNl4YOYScHu68PX0KAOHBJj4f+A6NOrSjYt3a5M6fjxZPdQFg5rgPuHTqzCPnMZvNfPnBR3z+01wMBgOrFv3FxbPneO3twZw+eoydGzdTsVpVPpo5gzz58tK4dSteGzqYPh272rsr7sliNrPw4+kMnDUNg9HI7iUrCPa/QOeBr3P5+CmObtlB9+EDyenhzuvTEi+yiAw2MWvwaLTFwpLPv2XID18BiisnTrNz0TKHZJo1aQrv/zATg9HAhkVLuXLOn+eHDODcsRPs3bSVstWqMO7baeTOm5d6rZrz/JABDOrcA6OLC5/8bx4AMTduMm3keCxms92Z7nAxGpn4+pO8PulHzBZNjzZ1KFfcl69+X0/VMkVpU78Se4+dZ9pv61BA3cqleK/fkw7bvhBCZHcqrXPiSqnDWusaSc+/BcK01u8nTR/SWtd8gG3oXpXqOiCqY/xxcj/Ny1RKv2Am2uZ/koFVHTey3V7fHtvJk+VrZnUMq2VnDsHxxVkdI1mVHlmdQAhnkKkDb6qUKOXwJvrxSxey5eCh9Lr6jUqpO70CbUgc1HdHtvjxHyGEEMJeOgMe2VV6lffvwFalVDgQA2wHUEqVBa6mtaIQQgghsp80K36t9WSl1EagELBOJ58XMJB4rl8IIYT4z8vOg/EcLd3ueq31nnvMe/TRc0IIIYTIMnKeXgghhNNznvZ++oP7hBBCCPEYkRa/EEIIp+dMLX6p+IUQQjg9ZxrcJ139QgghhBORFr8QQgin5zztfWnxCyGEEE5FWvxCCCGcnjO1+KXiF0II4fScqeJP8+58DuJM+1MIIYRjZOqd7UqVKOHwuurCpUvpvgelVAfgK8AIzNVaT7lreU7gZ6AOEAH01FpftCdXprT4y5SpmhmbeSD+/sfo9OzSrI5hY9Wf3ejcfWFWx7BaueQ5atXonNUxrA4eXknFEiWzOobVqUsXqVv7mayOYWP/v4uyOoIQ/2lZ0UJVShmBb4F2QACwTym1TGt9IkWxvkCU1rqsUqoX8CnQ057tyuA+IYQQImvUB85prc9rrW8DfwDd7irTDZif9HwR0EYpZVdviFT8QgghRAZQSvVTSu1P8eh3V5EiwJUU0wFJ8+5ZRmudAFwFCtqTSwb3CSGEEBkwpEBrPRuY7fAXtpO0+IUQQoisEQgUSzFdNGnePcsopVyAfCQO8ntkUvELIYQQWWMfUE4pVUoplQPoBSy7q8wyoE/S82eATdrOy/Gkq18IIYTIAlrrBKXUIGAtiZfzzdNaH1dKTQL2a62XAT8AvyilzgGRJB4c2EUqfiGEECJzfzbASmu9Clh117yJKZ7HAs86cpvS1S+EEEI4EWnxCyGEEFnU4s8KUvELIYQQzlPvS1e/EEII4UykxS+EEEI4UTs4Syv+iRPH0rJlM2JiYhk1ajzHj59MVaZq1cp89tlHuLm5sWXLdiZN+gSAr7/+nFKlSgKQN28erl27Tteuz1CkSGHWrVvG+fMXATh06AgTJkx66GxvvlqNerV9iIszM+3bg/hfuJqqzKTxDfHM74bRqDh+MoLvfjiCxQKlS+Zl0Bs1cM1hxGLWfDv3MGfORT90Bps8fWtRt44fcXFmps/Yi//51K83aUIzChRwT8oTzszZ/2KxaEYPb0jRInkAyJUrBzdv3mbwO+vtygMwavSbNGlal9jYON6bMJ1Tp/xTlRk46GW6dG1N3ry5adIo+cY2fn7eTProHfLkyYXBYGDGVz+xY8f+h9p+0xYtGP/eRAxGI4v+WMCcmTNtlrvmyMGn06ZRpVpVoqOieWfQIAIDAihStCgrN27ggv95AA4fPMj748fj5ubGlzO/o3jxEpgtZjZv2Mi0Tz99hD2T2oiRr9GkaS1iY2/z/nvfcPrUBZvlOd1y8Omnwyla1A+zxcL2bfv5ZsZvDtm2EEKklGbFr5TyAAaReOOiGSReP/g0cAqYpLW+8agbbtmyGSVLFqd1607UrFmdSZMm0KPH86nKTZo0gXHj3ufQoSPMmzeTFi2asnXrDoYMGWEtM3bsCK5fT45y+fIVunZ99Lun1a3lQ5FCuXh98EYqlCvAoDdqMGzctlTlPpm2n5iYBADGD69H04ZF2LYrkNderML//jzN/kOh1K3lw2svVmHM+zsfPU9tPwoXzs0bb62mQnlPBr5Zh3dGb0yd5/Pd1jzjRjWmaeOibNtxhU+/2GMt0/eVGty6Ff/IWe5o2rQuxYsXplvXN6hWrQLj3h3Iyy++k6rctq3/sOCP5SxdPsdm/utv9GL92u38+ecqSpcuxoxvPqBzp9ceePsGg4GJH07itRdexBQSwp/LlrFpw3r8z56zlnmm53Ncu3qVJ1q0pFPXrgwfM4Z3Bg0C4PKlS3Tv1CnV6/44ew7/7N6Nq6srP/7vN5q1bMn2LVseONe9NGlSi2LFC9G922CqVivH2LH9eKXP2FTlfvllGQf2H8fFxYWZs96jceNa7Np10K5tCyEejHKik/zp9W38BPgCpYCVQF1gKonDIGbef7X0tW3biiVLEn+g6NChI+TNmwdvby+bMt7eXuTOnYtDh44AsGTJMtq1a53qtTp37sCKFatSzX9UDesVYuPWxPsmnD4bRa5crhTInzNVuTuVrNGocHExcOfGjlqDh0fiMVUuD1cio2Lty1O/CJs2X0zMcyYyMU8Bt3Tz3Ou3nZo1KcbW7ZftygPQolVDVizfBMDRo6fJkycXXl4FUpU7evQ04eFRqeZrNLlyewCQO3cuwsIiH2r71WvW5PLFSwRcuUJ8fDyrli+nTbv2NmXatGvP34sXA7B21SoaNWmc5mvGxsbyz+7dAMTHx3Pi2HH8/PweKte9tGhZj1UrtgBw7OhZ8uTxoKBXfpsycbG3ObD/OAAJCQmcOnkeH1+77sMhhHgYSjn+kU2l19VfXmv9XNItAIOBtlprrZTaARy2Z8O+vr4EBYVYp0NCTPj5+RIWFm6d5+fnS0iIyTodHGzC19fX5nXq1atDeHgEFy8mV2ZFixZh2bI/uXHjBtOmzWD//n8fKpuXpxthETHW6fCIGLw83YmKjktV9sPxjShfNj8HDoWyY08QALN/OsqH7zai70tVUQYYMX77Q23/bgULuqfKU9DTnah7HFBMmticCuU82f9vMDt3B9gsq1LZi+joWIKCH7mjxsrHpyAhpjDrtMkUjo9PwXtW8vcya+ZvfPf9R/Tq3RV3dzf69xv/UNv39fMlODjIOh0SHEyNWjVtM/r5EhyUWMZsNnP9+nXyF0g8OClarBh/rVrJzes3+PLzzzmwb5/Nunny5qVV2zb8PG/eQ+W6F2+fgoSYkn9a2xQaiY93QSLCo+9ZPnduD5o1r8sfv6+0e9tCCHG3BzrHn1TZr7rz+8BJ03b9VrCjdO3aieXLk1v7YWFhNGvWjujoq1StWpnvv/+aDh26cePGzQzZ/oTJu3F1NTBqSB1qVPXm4JEwOrUvxZyfjrHzn2CaNSrM0AG1GP/hrgzZ/t0mTtqGq6uBkcMaUr2aD4cOJx84tWhW3CGtfUfo0LEFy5dt4Jefl1C9ekU+mjycZ3q8hZ0/Qf1AQkNDad2oMdHR0VSpWpVv5symS7v23LyReEBkNBr5YsbX/PLjTwRcuZLOqzmW0Whg8ifDWPDHKgIDQzN120I4M2fq6k+v4t+vlMqttb6htbaegFVKlQGu32+lpHsO9wOYNWuWdf6LL/aiZ8/Ec+9Hjx6jcGE/DhxIXHZ36x6SewHuKFTIF5MpuYzRaOSJJ9rSrdtz1nm3b8dz+3biQLxjx05w6dIVSpUqydGjx9N8o12eKMUTbUsAcPZcFN4F3a3LvAq6Ex4Zc79ViY+3sHtfCA3r+XHwSBhtWxZj1o9HAdi+O4ih/Wumue176dyxLB3alQLgzD3yRKSTZ8/eQBrWL2yt+A0GReOGRRk64tEH9T3XszNPP90BgOPHz+Dn621d5uvrRWjog98w6qnu7Rk4IPFXKY8cOUWOnDnIXyAvUZGpB1HeiynERKFCha3TfoUKYbrr+xMaYqJQ4cKYQkIwGo3kyZOH6KjEHono27cT38exY1y5dJlSpUpx7GjiZzZpyidcunDBrtb+s8914KnubQA4cdwfP9+C1i4yXx9PQsPuva/Gv9ufK5eD+f1/0toXQmSM9M7xzwJy35lQSr2slFoKvA08db+VtNaztdZ1tdZ1+/XrZ53/669/0LXrM3Tt+gzr1m2ie/cnAahZszrXr9+w6eYHCAsL58aNm9SsWR2A7t2fZMOGzdblTZo0xN//vM0Bg6dnAQyGxLdVrFhRSpYszuXL6bfaVqy9wOCRWxg8cgu794XQpkXinRIrlCvAzVvxqbr53dyM1vP+BoOifh1frgQmthgjImOpVjnx/GyNql4Ehjx8b8PK1ecY/M56Br+znj3/BNK6VcnEPOU9E/Pc1c3v5uZiPe9vMCjq1SlEQEDysVmtGr4EBF4jIuL+BwzpWbhgJb16DqZXz8Fs3ryHLl0Tx1tUq1aBGzduPnA3P0BIcBj1G9QEoFSpYuTM4frAlT7A0cOHKVGqJEWKFcXV1ZVOXbuyab3tQc2mDet5qkcPAJ7o1Ik9uxJ7XQp4elq/I0WLFaNEqZJcuZzYEzJ0xHDy5MnDxx88/JUgKf25cA0v9B7JC71HsmXLXjp1aQlA1WrluHHj1j27+Qe81YvcuT344vMf7dq2EOJRGDLgkT2l1+KfBbQFUEo1B6YAg4GawPck3iLwkWzZso2WLZuxadNqYmNjGD16gnXZ8uWLrKPy33vvIz777CNy5nRj69btbNmSfL68S5eOLF++2uZ169Wrw9tvDyIhIQGLxcKECZO4evXaQ2Xb96+JerV8+WFGW+Jum5n+bfLI6hlTWzJ45Bbccrrw3ugGuLoaUEpx5Hg4q9ZdBODrWYd489VqGA2K+HgLM2Ydesi9c1eeA8HUrVOIuTM7EReXwPQZyeejZ0xrx+B31uOW08jEsU0T8xgUR4+Gsmpt8uV1zZsWY+t2x3Vb79i+j6ZN67JsxVxiY+N4f+J067I/FsygV8/BAAx9+1U6dmqJm1tO1qybz5K/1jLr+/8x7Yu5TJg4hBdf7IbWMDHF+g/CbDbz4cSJ/PDzzxiMRhYvXMi5s2cZ/M4wjh05yuYNG1i0YCGfTZ/G2q1buBodzTuDEjPVa1Cfwe+8Q0J8AhZt4f1x47l69Sq+fn4MGDwY/3Pn+GtlYov7t5/ns+iPBXbtq507/qVJ09r8vfQbYmPj+OD976zLfvt9Ki/0HomPjyd9X3+GCxcC+PV/nwGwcMEalv6d+uoNIYTjqWw8GM/RVFrnVJVSh7XWNZKefwuEaa3fT5o+pLWu+QDb0GXKVHVAVMfw9z9Gp2eXZnUMG6v+7Ebn7guzOobVyiXPUatG56yOYXXw8EoqliiZ1TGsTl26SN3aj365aEbY/++irI4ghKNlak1cpnRlhw8w8j9/IlseTaTX4jcqpVy01glAG5LO2z/gukIIIcR/g8q+XfOOll7l/TuwVSkVDsQA2wGUUmWBBz8hK4QQQohsIc2KX2s9WSm1ESgErNPJ5wUMJJ7rF0IIIf7zVDYejOdo6XbXa6333GPemYyJI4QQQoiM5DyHOEIIIYSQAXpCCCGEM13OJy1+IYQQwolIi18IIYRwosv5nOedCiGEEEJa/EIIIYRyoha/VPxCCCGcnjNdx+8871QIIYQQ0uIXQgghnKmr33neqRBCCCHSvi2vg2T4BoQQQjx2MvUXdSpVbOzwuurkqV3Z8leBMqWr/9mKdTJjMw/kz1MHmFK9TVbHsDHmyEYW1Wqb1TGsnjm4gf5VGmd1DKvvj+/iRJf6WR3DqvKKvUSMbZrVMWwU/GQH7206mNUxrD5oXSurIwjxUKSrXwghhBCPJRncJ4QQwulJi18IIYQQjyVp8QshhHB6ShmzOkKmkRa/EEII4USkxS+EEMLpOdM5fqn4hRBCOD3p6hdCCCHEY0kqfiGEEMKJSMUvhBBCOBE5xy+EEMLpOdM5/mxX8dds2ohXx4/AYDCycdHf/D3nJ5vlXV55gTbPPIXZbOZaZBTfjf+A8KAQh2Yo1aQebUcPxGAwcPivVeyZ94dtxme7ULtXN7TZwu1bMayZNJ2I85coVLUCHSa+A4BSih0z53Nm00678/g2rkfNkW+hDAYu/L2a0z/+cc9yRdo0o9Hn77HxhbeIOnGGHPny0nDqRDyrVODisrUc+vQbu7PcUblpA54b8zYGo5Gdi5ezdu4vNsvb9OlF0x5dMSeYuREVzc/vfkxkcOLnVKCQLy99MJYCfj6A5pv+w4mw8zPMVbshfv2GowwGotYtJWLRzzbL87XpjO9rQ0iICAMgcsWfRK9bal1ucM9FmZl/cH3PVkK+/9yuLACu5RuQq8tQMBiI3beC2K2/piqTo1pr3Nu8CoA5+Bw3FnyQmCWfL7l6jMaQzwe05vpPI7FE27d/tNb8u3A+wccPYsyRkwYvD8CzeCmbMvGxMWz84n3r9K2oSErWb0rt5/pY51359x92zplO+zGT8SxRxq5MQmQnBhnVf39KqTNa6/IZEcZgMNB34hg+fO0tIk0mPvnzF/Zv2kqA/wVrmQsnTzP6mZe4HRtL+17P8NKIoUx/Z6zDMiiDgfbjhvBHv1FcN4Xxyu/fcXbLbiLOX7KWObFqE4f+XAFA2ZaNaDOyPwsHjCXs3EV+6j0AbbaQy8uT1xbN5uzW3Wiz5dEDGQzUGjOY7QNGc8sURpvfviVo6y6un79sU8zFw52yz3cn4shJ6zxz3G2Of/cT+cqWJG+Zko+e4S7KYKD3+BF89cZQokyhjF3wA0c2byfY/6K1zJWTZ/j4udeIj42jec/uPD38LeaOmAjAqx9PYPXs+ZzcvY+cHu5YLHbsHwCDgUIDRnHp3UHER4RSevp8rv+zndtXLtgUu7Z9/X0rde+X3uTWsUP25bhDGcj15Dtc+2EYlmuh5Bs4l/iTOzCHXkyOXLAo7i1f5Nr3b6Fjr6Ny5bcuy/3cu8Rsnk/8uf2Qwx20nfsHCD5+iBuhwXT+4EsiLpxj/+9zaT96sk0ZVzd3Ooz/1Dq99uOxFK2VfHOk+NgYzmxeTcGSZe3OI4TIOmke4iilriulriU9riulrgNl7sx3dJiy1asQcvkKoQGBJMQnsHPVOuq2aWlT5vg/+7kdGwvAmcNH8fTzcWiGQlUrEnU5kKuBwVgSEjixZjPlWtneqe72zVvW567ubty5s3FCbJy1knfJmcMhNyT2rFqBG1eCuBkYjE5I4MraLRRu2SRVuSpvvcLpHxdguX3bOs8cG0vEoWOY426nKm+PktUqE3olgPCAIMzxCexbtYHqrZrZlDmz91/iY+MAuHD4eFLrHgqVKYnBxcjJ3fsAiLsVYy33qNzLV+F2cADxpiBISODqtnXkadj8gdd3K1MRl/ye3Di4x64cd7gUq4Q5IgBLVBCYE4g7vAHXSrZ383Or15XY3X+hY68DoG9GA2D0KQkGY2KlD3A7BuLt2z8AgYf3U7Jhc5RSeJUuR/ytW8Rcjbpv+WumIGJvXMW7bEXrvKPLFlKp/ZMYXF3tziNEdqOU0eGP7Cq9vo0fgb+BclrrPFrrPMDlpOd5HR3G09eHiGCTdToyxERBX+/7lm/zTDcObtvl0Ax5fL24bgqzTl83hZHHxytVudo9u/Hmyl9oNawfG6Ykd6EXqlaRvn/9QN/Fc1n74XT7WvuAu48XMaZQ63SMKQx374I2ZfJXLIu7nw8hO/6xa1sPqoCvN1EpPqdoUxgF0vicmvTowrHtiZWqT4ni3Lp2gze//Jhxi37i6eEDUQb7uthcCnoTH5acJyE8FNeCqfPkadya0jN+o+jYT3DxSjpgVArf14di+uFruzKkZMjrjeVq8mdmuRaGMZ9tHqNXMYxexcj75nfkHTAL1/INEtf1KoaOvU7uFyaTb/A8PDq+BQ7ogoyJjsSjQPL3xr2AJzHRkfctf3n/borXaYRSibcTj7x8gVtRERSuVtvuLEKIB6OU8lRKrVdKnU36t8A9ytRUSu1WSh1XSh1RSvVM73XT/IuitR4CfAX8rpQaohJ/2sgB7Vj7NevakdJVKrPsh5/TL5wB/l2wlFmdX2LLl3No3O9F6/zgo6f44em+zO/9Fg37Po8xRwa3jpSixvABHPni+4zdziOq3+UJilepyPp5vwFgdDFSrk4NFn/+DVN69sWrWGEaPdUpw3Pc2LuDc6914/zgF7hxcC9Fhr0PQIHOz3Bj/y4SIkLTfgFHMxoxehXj2pzB3PjjfXJ1H4Vyy40yGHEpWYNbq77l6rdvYPAsTM46HTM3G3B5/y5K1E3sWdIWCwcX/UzNZ15MZy0h/ruyaYt/DLBRa10O2Jg0fbdbwMta6ypAB+BLpVT+tF403XP8WusDSqm2wCBgK+CW3jpKqX5AP4BZs2alV9wq0hRKwUK+1mlPP18iUrS+76jWqD5P9+/Ley+9QUJ8/AO//oO4bgonT4rWax5fb66Hht+3/InVm2k/fmiq+REXLhMfE4N32VKEnDjzyHliQsNx900+neHu601MWIR12iWXB3nLlKTF3C8AcCvoSeMvJ7Hr7YlE2bHdtESZwiiQ4nPK7+tN1D0+p4oN69KxXx+mvTLQ+jlFhYRy5dRZwgOCADi8cTulalRh118rHjlPQkQYrt7JeVy8fIiPsM1jvn7V+jx63VJ8Xx0MgEfFanhUrkmBTj0wuHmgXF2wxMQQOv/bR85juRaWODAviSGvN+artnksV8NIuHICLGYsUcFYwq9g8CqK5WoY5qCziacJgNsntuNarApxrHzoHGe3rMV/5yYAPEuU4VZU8vcmJioS9/ye91wvKuAS2mLGs0RpAOLjYrkaFMCmaZMAiL12lW0zP6f5gBEywE88NrJp13w3oGXS8/nAFmB0ygJa6zMpngcppUIBbyD6fi+a3jn+ekopP621RWv9NbAK8FJKfaWUuvdfjcSNz9Za19Va1+3Xr1+a7yqlc0dPUKhEMXyKFMbF1YUmndqzf9NWmzIlK1Wg3wfj+fStYVyLvP85ykcVfPwUniWKkK+IHwYXFyp3aMW5LbanEwoUL2J9XrZ5Q6IuBwKQr4gfypi4S/MW8sGzZDGu2jlaPer4aXIXL4JHYT+UiwvFnmhJcIo8CTdusrx1D1Z3fpHVnV8k8ujJDK30AS4dO4lP8aIULFIIo6sL9Tq15cjmHTZlilUszwvvjWbmoFFcT/E5XTx2Eo+8ucldID8AFRrUIdjfdhDew4o5c4IchYvh6lsYXFzI17w9N/7ZblPGJUU3d54GzYlLGvgX+PlEzr72JOf6PoVp3ldc3bTKrkofICHgFEavYhgKFAKjCzlrtCX+pO3VHbdPbMeldC0AlEc+DF7FsEQGkRBwEuWexzrYz7V0bRJSDAp8GOVaPkGH8Z/SYfynFK1Rl4t7tqG1Jvz8WVzdPXDPl6rXEIDL+3ZSvG7yOJIc7h48/fkcnpz8DU9O/oaCpcpKpS9E5vDVWgcnPQ8BfNMqrJSqD+QA/NMql16LfxbQNukFmwODgd5ATWA28Ex6qR+GxWzmhw8/Y/wP32AwGNm8eCkB587Tc3B//I+dYP/mbbw0cihuHu4M/zJx9HF4cAifvvWOwzJos4V1H8+g58xPUUYDR/5eTbj/JZq99QrBJ05zbstu6vR+ihINamNJSCD22g1WvpuYpWitqjR8rTeWhAS01qyb/DUx0faNgdRmC4c+nUGz76agDAYuLl3DtfOXqDygD1EnzhC8dXea63dc+SuuuTwwuLpSuFUTtr81OtUVAQ/LYjazYPI0hsyejsFgZNeSFQT7X6DroNe5dPwURzbv4OkRA8np4c4b0z8CIDLYxMxBo9EWC4unfsPbP3yNUorLJ06xY9Eyu/JgMRPy/VSKT/oaZTAQvX45cZfP4/1CP2LOnuTG3u14PtmT3PWbg8WM+fpVgr6cZN8208lzc9k08r42DZSBuP0rMYdewL1tXxICTxF/cifxZ/7BtVw98r39C2gLt1Z/h76V+F25teob8vb9EpQiIfA0cfvs3D9Aoaq1CDp2iBUTh+KSIycNXu5vXbZm8mib0fyXD+yhxaDR93oZIR5byuD4Fn/K3u8ks7XWs+8qswHwu8fq41NOaK21Uuq+p9qVUoWAX4A+Wqd9KZDS+v6n7JVSh7XWNZKefwuEaa3fT5o+pLWumdaL38n7bMU6D1Asc/x56gBTqrfJ6hg2xhzZyKJabbM6htUzBzfQv0rj9Atmku+P7+JEl/rpF8wklVfsJWJs0/QLZqKCn+zgvU0HszqG1Qeta2V1BPHfpzJzY/Xq9nT4+LV9+xfY9R6UUqeBllrr4KSKfYvWusI9yuUl8TTAx1rrRem9bnrDhY1KqTu9Am2ATSmWZbsf/xFCCCEehUEZHf5wgGXAnV/Q6gMsvbuAUioHsAT4+UEqfUi/4v8d2KqUWgrEANuTNlQWuJrWikIIIYSwyxSgnVLqLImn3acAKKXqKqXmJpV5DmgOvKKUOpT0qJnWi6bZatdaT1ZKbQQKAet08nkBA4nn+4UQQoj/vOw4ql9rHUFib/vd8/cDryc9/xVI/ZvgaXiQy/lS/ZxZyssHhBBCiP+67FjxZxTnuSuBEEIIIWSAnhBCCJE8jv3xJy1+IYQQwok4zyGOEEIIcR8OuvzuP0Fa/EIIIYQTkYpfCCGEcCLS1S+EEMLpZcRv9WdX0uIXQgghnIi0+IUQQjg9Z7qcz3neqRBCCHEfzvTLfWneltdBMnwDQgghHjuZelveFs2GObyu2rp9eqa+hweVKS3+YVWbZcZmHsj0Y9tpXbZyVsewsencCVbXa53VMaw67tvELzXbZXUMq5cOredg20ZZHcOq1obdTKqefT4vgIlHNtGx9+qsjmG1+veOBPevl9UxrAp9vy+rI4hszpm6+mVwnxBCCOFEnOcQRwghhLgPZ/rlPqn4hRBCOD1lcJ7qULr6hRBCCCfiPIc4QgghxH3I4D4hhBBCPJac5xBHCCGEuA9n+gEfafELIYQQTkRa/EIIIZyeM53jd553KoQQQtyHXM4nhBBCiMdStjjEqdikPt3HDEUZDfyzeAUbf/jNZnmLl3vSsEcXLGYzNyKj+WPCJ0QFmyhbrxZPjR5sLedTqjg/j/yAY5u225WnXvOmDHp3LAajkVULF/H7rLk2y6vXq8PAd8dSukJ5Pnx7BNvWrLMuW3/6KBdOnwUgNDiId98cZFcWAK9G9ag0fBDKYCBg6SrOz//9nuV8WzWj9mcfsPPl/lw7eQb3Qr40W/gTNy9fASD66AmOT/nS7jwAhRvXpe6ot1AGA+eWrOb4jwtslpd7pgsVej6JtlhIuBXDng+nc/X8ZQwuLjSY8DYFK5dHWyzsn/odpv1H7M6Tp15Dir71NspgJGL1Mkx//GKz3LN9Jwr3G0R8eBgA4UsXEbF6Oe5lylFs6EgMHrnAYiHkfz8RvWWj3XnKNKnHE6MHYTAYOPjXKnbOs/3M6jzblbq9uqHNFm7fimHFpGmEn79E6YZ1aP32GxhdXTDHJ7Bh2iwu7j1od5576d+nEvVqehN328wXM4/if/Hafcu+N6I2fj4eDBi1w2Hbz1m5EXmfGw4GA7d2LuXm2vmpyrjVaUvuLm+AhoSAM0TPm4BL0fLke340yi03WMzcWP0jsQfWOyyXcE7S1Z+JlMFAj3ff4fs3hhEdEsawBXM4tnknpvMXrWUCT55hWs/XiY+No3HPp+g6fAA/j3ifc/sO8vkzrwHgkTcP41b/welde+3KYzAYGPr+u4zs8zphISZm/rWAXRs3c+mcv7WMKSiYT0eN47nXX021/u3YOPo9+bRdGe4KRJVRQ9k7aCSxpjAaz59J6LZd3LhwyaaY0cOdkr16EH30hM38W4FB7Hyhn+PykPiZ1R87mA39R3PLFE7H374hYOturp6/bC1zcfUmzi5aAUDRFo2oM7w/mwaOo2yPTgCseLYfbgXy0/rbyax6YRDYc5dIg4Fig4dzbvRQ4sNCqfDtPK7u2k7s5Ys2xaK3bCTgmy9s5lliY7n06STiAgNwKehFxe9+5Pq+fzDfvPHIcZTBQMdxQ/m130iumcJ4/feZnN6yi/DzyZ/Z0VUbOfDncgDKt2xM+5ED+N+AMdyKvsofg8dzIywC77IleWHmZ3zZ7rlHznI/9Wp6U9gvF32HbaNi2fwM6luFYRN237Ns43q+xMSaHRtAGcjbexSRXw3CHGXCa+x84o5sIyH4grWI0acYuZ94hYipr6NvXceQpwAA+nYs0T+9jzn0CoZ8XniN+4W4E7vRMY/+mQnhTNLs6ldKVU/x3FUp9a5SaplS6mOllIcjAhSvVonwy4FEBARjTkjg4OqNVG3d1KbMuX0HiY+NA+DS4ePk9/VJ9To12rfk1PY91nKPqmKNagReukzwlQAS4uPZtHI1jdva3onNFBjE+dNnsFgsdm3rQeSvUpGbVwKJCQxGJyQQvH4TPi0apypXvv9rnP/5d8y3b2d4poJVK3D9ShA3AkOwJCRwae0WirW0zRR/85b1uYu7m7Viz1+6BCF7DwEQGxXN7es3KVilvF15PCpUJi4ogNvBQeiEBKK2bCBfk+YPtG5c4BXiAgMASIgIJyE6Cpf8+e3KU6RqRaIuBxIdGIwlIYHjazZRoZXt/rmdYv+4ptg/IafOcSMsAoCwcxdxdcuB0dXVrjz30rCODxu3BwJw6lw0uT1cKJA/Z6pybjmNPN2pJH8s8U+1zB6uJatgDr2COTwQzAnE7FtPzuotbMp4NH2Km1v/RN+6DoDlehQA5tDLmEMTe7EsV8OxXI+0HhQIIdKX3jn+n1I8nwKUBb4A3IHvHREgv4830SGh1umrpjDy+Xjdt3yDpztzcvueVPNrdWzDv6vt76L18vUlNDjEOh0eEoL3PQ407idHzhzMXLKQbxb9TpO2bezO4+btRawpef/EmsJx8/a2KZO3QjncfL0J2/lPqvXdC/vR5NdZNJg1nQI1q9mdB8DDx4ubIWHW6ZumcNzv8ZmV7/kkTy2fT+23X2ffZ98BEHXGn2ItG6GMBnIX9qNg5XLk8vVOte7DyOHlze3Q5H10OywU14KpXzN/s5ZUnP0LJSdOxtU79WfqUaEyysWVuKBAu/Lk8fXiaorP7JopnDw+qfPU7dmNQSt/pe2wfqyZ8k2q5ZXaNSf45FnM8fF25bmXgp5uhEfEWqfDI2Px8kxd8b/8XDn+WnmR2DjHtviNBbwxR5ms05ZoE8YCtvvIxac4Lr7FKThyLgVHzSNn5dS3ZnYtWRlldMUcFuDQfMIJKRfHP7Kp9JKpFM/bAPW01vFKqW3A4YyLdW91urSnWJWKfPPKYJv5eb0KUqhcGU7do+LLbL1btCXcFEqhYkX54pcfuXDmDEFJ59gzhFJUHDaAox98mmpRXHgkW7r2Jv7qNfJWLEftzz9kR8/XSEjR2sxIZxYs48yCZZTs2IpqbzzPrglTOff3GvKVKk6n/33HzSATYYdPoDOh5+Tqnh1EbV6Pjo+nYOenKDFqAudGJn+PXDwLUmLMRC599qF9px0ewv4FS9m/YClVO7WmWb8XWfpu8mfoXaYkbd7ux29vjsqULPdSukQeCvl6MPuXU/h4uWd+AIMRF59iRHzxJsYCvhQcPpuwD3tZu/QNeQuS/5VJRM9/P9M+MyEeB+lV/PmUUt1J7BnIqbWOB9Baa6XUff+nKaX6Af0AZs2aleYGokPDyO+X3PrK5+vN1dDwVOXKN6xDu34v8c0rg1O1gGp2aMXRjduwJNjfKgk3mfAp5Ged9vLzIyxF6y399RPLBl8J4NA/eylbuZJdFX9sWDhuKXoc3Hy9iA1Lbm27eHiQp0wp6n8/HYCcBT2p88VHHBj+LtdOnsFyNXFfXTt1llsBQXgUL8q1k2ceOQ/ArdBwcvklt85y+XoRc4/P7I6La7bQYNxQYCrabGH/58mdRU/M/5Jrl+xrrd0ODyOHT/I+yuHtQ3xEmE0Z87XkgWsRq5dRpN9A67TBw4Myk78geN4sbp08blcWgOumcPKl+Mzy+npxPTTsvuWPrd5Mp/FvW6fz+Hrx3PQPWDr+E6ICguzOc0eXdsXp0LoYAGfOX8WroJt1mZenG+GRtqfJKpUrQLnS+fjp6xYYDQby5cvBpxPqM/pD+8bRAJijwjAW8LVOG/L7Yo666zOLDiX+wnGwmDFHBJEQehkXn+LEXzqBcsuF56Avub7sO+IvHLM7jxByOV+yrcCTQBdgj1LKF0Ap5Qfc9y+91nq21rqu1rpuv35pDyy7cuwU3sWL4lmkEEYXF2p1bMPxzbYjh4tULMez741k7qCx3IiMTvUatTq25d9VG9J5Kw/m1JFjFClRAr+iRXBxdaV1547s3rj5gdbNnTcvrjkSz8fmLZCfqnVq2wwKfBRXT5wiV/EiuBf2Q7m4UKhda0K3JQ/CSrh5k43turO12/Ns7fY80cdOWCv9HPnzgSHxI3YvUohcxYoSExhsVx6AiOOnyVO8CLkL+2FwcaHEEy25stV2YFie4kWsz4s2a8D1y4nd50a3nLi4JVY4hRrWRieYbQYFPopbp0+Ss0gxcvgVQrm4UKBlW67usr2yw8WzoPV5vkbNrAP/lIsLpd//lMj1q4ne/mCfc3oCj5/Cs0QR8hdJ3D9VOrTmzBbb/eOZYv+Ub96QyKT9kzNPLnp/8wkbv5rLlUP2H4SktGL9ZQaN3cmgsTvZvd9Em2aJGSqWzc/NWwlERdtW/Cs3XObFtzbzypCtDH9/D4HBNx1S6QPEXzqB0ac4xoKFweiCe712xB3ZZlMm7tBWcpSvDYDKlQ8Xn+IkhAeC0YUC/adya88qYv/d5JA8QjiT9A5xvgMCtNbBAEqpl5VSPYBLwLOOCGAxm1n88XTenPUFBqOBf5asJMT/Ih0G9uXK8VMc37KTJ4e/RU4Pd16ZNgmAqGATPwweC0CBwn7k9/PBf/8hR8TBYjYz44PJfPrjHIxGA6v/XMLFs+d4Zeggzhw7zq6Nm6lQrSqTZn5N7rx5adS6Fa8MHcRrHZ+kRJnSDPvofbTFgjIY+H3WHLsrfm22cOKzGdT7+lOU0UjAstXcOH+Rcm++wtWTZwjdtuu+6xaoVZ1y/V9FJySgLZrjU6YTf+26XXnuZNo75RvazPwk8XK+pWu56n+JGgP6EHHiDAFbd1OhVzcKNaiFJcHM7WvX2TnxMwDcPPPT5rtPwKK5FRrOzndTn6J4aBYzATO+oMyUL1EGAxFrVhB76QJ+fd7g1pmTXNu9A+/uz5GvUVMwm0m4fo1Ln30EQP4WbchdvSbGvHnxbJ94xcHlqR8R43/Wrv2z+uMZvDAz8TM79Pdqwvwv0vKtVwg6cYYzW3ZRr/dTlGpQB0tCArHXrlu7+ev36o5n8cI0f/Mlmr/5EgC/9h/FrXsc8Npj38Ew6tX0Zt6XLYiNMzN9VvIlld980oRBY3c6dHupWMxcW/AZnkO+BoORmF3LSAg+T+6ubxJ/6SRxR7YRd2I3OSo3wOu9BWCxcO2vr9A3r+JevyM5ytXCkCsfHo26ABA9/wMSAuzryRLOzZku51M6jXNjSql/gbZa60ilVHPgD2AwUBOopLV+5gG2oYdVbeaIrA4x/dh2WpetnNUxbGw6d4LV9VqnXzCTdNy3iV9qtsvqGFYvHVrPwbapB3ZllVobdjOpevb5vAAmHtlEx96rszqG1erfOxLcv15Wx7Aq9P2+rI4gHp5Kv4jjdOr6o8MHiqxa/mqmvocHld4hjlFrHZn0vCcwW2u9GFislDqUocmEEEII4XDpVvxKKRetdQKJo/pTnrB3nn4RIYQQjzcnui1vepX378BWpVQ4EANsB1BKlQWuZnA2IYQQQjhYmhW/1nqyUmojUAhYp5MHBBhIPNcvhBBC/Oc50+V86b5TrXWqn8nTWsvwWSGEEOI/yHkOcYQQQoj7caLL+ZznnQohhBD3oZ2oqz+9X+4TQgghxGPEeQ5xhBBCiPsxOM/lfNLiF0IIIZyItPiFEEIIafELIYQQIisppTyVUuuVUmeT/i2QRtm8SqkApdQ36b2uVPxCCCFE9jQG2Ki1LgdsTJq+nw+BbWkst5KKXwghhNPTBqPDHw7QDZif9Hw+8NS9Ciml6gC+wLoHedE0b8vrIBm+ASGEEI+dTL2lbcfeqx1eV63+vaNd70EpFa21zp/0XAFRd6ZTlDEAm4AXgbZAXa31oLReN1MG942t1jIzNvNAPjm6hV6V6mZ1DBt/nNzPTzXbZXUMq1cOrc92eSZXb5PVMazGH9nIqGotsjqGjc+ObqV1i7R6ATPXpq1TWFSrbVbHsHrm4AY2N8w+n1mrPVuzOoK4i4Na6DaUUv2wvavtbK317LvKbAD87rH6eJt8Wmul1L0OTt4CVmmtAxKPDdIno/qFEEKIDKj4kyr52emUue8RslLKpJQqpLUOVkoVAkLvUawR0Ewp9RaQG8ihlLqhtb5vS0AqfiGEECJ7Wgb0AaYk/bv07gJa6xfuPFdKvUJiV3+a3X8yuE8IIYTT0waDwx8OMAVop5Q6S+L5+ykASqm6Sqm5j/qi0uIXQgghsiGtdQSQaoCT1no/8Po95v8E/JTe60rFL4QQwullxOC+7Eq6+oUQQggnIi1+IYQQTs9idJ52sFT8QgghnJ6DBuP9JzjPOxVCCCGEtPiFEEIIafELIYQQ4rEkLX4hhBBOz+JELf5sV/GXb1KfLqMHYTAa2ffXSrb+8D+b5fWffZJGvZ/CYrZw+1YMSz74nNDzlxyaoUbTRvQZNwKDwcCmRX+zbO58m+Wd+rxA62e6YTabuR4ZxffvTiI8KASAMbO/plyNapz+9xCfDRjmkDxFGtel/qi3UAYDZ5es5uiPC2yWV3imCxV7Pom2WIi/FcOuD6dz9fxlDC4uNJrwNl6Vy6MtFvZO/Y6Q/Ucey0ylm9Sj/eiBKIOBQ3+tYve8P2yW1362C3V6dUMnfW9WTZpO+PlLFK5agU4T30kspBTbZ87n9Kadducp36Q+3UYPRhkN7P1rJVvu+h43e/k56j/dGYvZzI3IaP6c+CnRwSYAOg57k0rNGgKwcdbPHF672e48AIOGdKVBgwrExsXz2Sd/cvZskM3ynDldee+DFyhc2BOLRbN710nmzF4DwFsDu1CzVunEcm6uFMifmye7fGBXHt/G9ag5MvE7dOHv1Zz+8Y97livSphmNPn+PjS+8RdSJM/g0qE21Ia9jcHXFEh/PkS9nE7bvkF1ZADwb1qfcsMFgMBC8bCWXf/nfPct5t2pO1U8+ZP8r/bh+6jQAucqWpsLoEbjk8kBbNAdeexPL7dt2ZxKZR8uo/kRKqUHAH1rrcKVUWWAeUB04DbyutT7qyDDKYODJ8UP5od8IroWEMfCP7zm5eadNxX541Qb2/rkMgEotG9N55EB+HDDKoRlemzCayX0HEmEy8fHCnzmweRuB/hesZS6ePMW4ZxdxOzaOdr168MKIIXz1zjgAVsz7hRxubrTt+bTD8jQYO5h1/UdzyxROl9++4fLW3Vw9f9la5vzqTZxetAKAYi0aUX94f9YPHEf5Hp0AWPpsP9wK5Kftt5NZ8cIgsPNWzNktkzIY6DBuCP/rN4prpjBe+/07zm7ZTXiK782xVZv498/EPOVaNqLtyP78MWAsoecu8kPvAWizhdxenry+aDZntu5Gmy125ek+/m3m9BvO1ZAwBv8xixN3fY+DTp7l6179iI+No+Fz3ej8Tn9+G/kBFZs1pEil8nz57OsYc7jSf95XnNrxD3E3bz1yHoAGDSpQpKgXL73wOZUqF+Ptd55i4IDvUpVbuGAbhw6ex8XFyOfTX6d+g/Ls/ecM3327wlqm+9ONKVuusF15MBioNWYw2weM5pYpjDa/fUvQ1l1cT/EdAnDxcKfs892JOHLSOu929DV2vj2B2LAI8pYpSbPvprDyiV525yk/4m0ODRlOXGgYdX+cRfj2ndy6aNuoMHq4U/S5Z7h67Lh1njIaqfz+u5x4fzI3z/njkjcvloQE+/IIkYHSO8QZoLUOT3r+FTA96V7Ao4HvHR2mWLWKRFwOJCogGHNCAodXb6JSqyY2ZVL+Aczh7obGsbdQLlu9CiGXrxAaEIg5PoFdq9ZRt7Xt7TxP7D3A7dg4AM4ePoanr6912bE9+4i18490Sl5VK3D9ShA3AkOwJCRwYe0WirdsbFMmPsX2XNzd0EmVaL7SJQjeewiA2Khobl+/iVeV8o9dpsJVKxJ5OZDowGAsCQmcWLOZ8q1s89xOkcfV3Y07X5uE2DhrJW/MmcPeYyIAilWrRPjlQCJTfI+rtGpqU8Z/30Hik75Dl4+cIJ+vNwC+ZUpy4cBhLGYz8TGxBJ/xp0LTBnZnaty0MuvX/gvAyRNXyJ3bHU/PPDZl4uLiOXTwPAAJCWbOngnC2ztfqtdq3aYGmzYesiuPZ9UK3LgSxM3AYHRCAlfWbqFwyyapylV56xVO/7jApvUcffocsWERAFzzv4gxZw4Mrq525clbuRIxAYHEBiXmMa3fhFfzpqnKlerXl8u//M8mT4H6dblxzp+b5/wBSLh2DSyPfuAosoY2KIc/sqv0Kv6UPQI+WuslAFrrLUCee65hh7w+3lwNCbNOXzOFWf8gptSw11OMWPUbHd7pz/JPvnZoBk8fHyJCTNbpSFMonr4+9y3fqkc3Dm3f5dAMKXn4eHEzxT65aQrHw8crVbmKPZ/k6eXzqfv26/zzWWJLLuqMP8VbNkIZDeQu7IdX5XLkusf+/K9nyuPrxXWT7fcmzz3y1OnZjbdW/kKbYf1YO+Ub6/zC1SrS768f6Ld4Lms+nG5Xax8gn48XV0OS75551RRGXt/Uee6o93QnTu34B4Cg0+eo0KQ+rm458cifjzL1a5HfAZ+Zl1deQkOjrdNhYVfx8s573/K5crvRqHFF/j3gbzPf1zc/foUKcPBf//us+WDcfbyIMSXvoxhTGO7eBW3K5K9YFnc/H0KS9s29FGnbjKhT57DEx9uVJ6e3F7GhyXniQsPI6W37meWuUI6cvj5E7NpjM9+jeDHQUOPLqdSdP4fiL/a2K4sQGS29in+RUuonpVRpYIlS6m2lVAml1KvA5futpJTqp5Tar5TaP3t2mrcifiR7/vibzzu9wJrps2jd7yWHv/6Datq1I6WrVmL5Dz9nWYY7Ti1Yxl9d+7D/q7nUeON5AM7+vYabpjC6/u876o8cQOjhE+hMbIlkt0wHFizlu84vsenLOTTt96J1ftDRU8x+ui/zer9F477PY8xhX+vxYdTq0o6ilSuwNen89tnd+zm1fQ8Df/mW5z+byOXDx7FkcuvRYDTw7sTeLFm8i+DgSJtlrVrXYNvWY1gsju1pS0UpagwfwJEv7t+xmLd0CaoNeYN/P5qesVmS8pQdOhD/r1OfHlFGI/lqVOPEex/xb79BeLVoRoG6tTM+kxCPKM1z/Frr8Un39/0dKAPkBPoBfwMvpLHebOBOja/Hzrj3IJm7XQsNI59fcusmr683V1O05O52ZPUmnnrXMQPo7ogMDaWgX3LXvaevD5EpWiZ3VG1Un+5vvsYHL/cjwc7WRlpuhYaTK8U+yeXrxa3Q8PuWv7BmC43GDQWmos0W9n2e/Iez0/wvuXop4LHLdN0UTh5f2+/N9TTyHF+9mQ7jh6aaH3HhMrdjYvApW4rgE2ceOc/V0HDy+SX3EuXz9eaaKXWesg3r0PqNl/j+1SGYU3yHNs35lU1zfgWg96cTCL905ZFydHuqIZ271Afg9OkAfHzyA4nnrL298xEedu2e6w0f8TSBAeEsXpR6kGOrNjX4evrfj5QnpZjQcNxT9KS5+3oTk9R9D+CSy4O8ZUrSYu4XALgV9KTxl5PY9fZEok6cwd3Hi0bTPmDfhE+5GRBsd564sHDcfJLz5PTxJi4s+TMzeniQq3Qpan73JQA5PD2pNvVjjo4cR1xoGNEHDxN/9SoAEbv2kLtCeaL2/2t3LpF5LMbs2zXvaGm2+JVS9YA1WusGWmsvYBBwDsgNOPxWRgHHTuNVoigFivhhdHGhRsfWnNxi241esHgR6/MKzRsSfjnQoRn8j57Ar0QxvIsUxujqQuNO7TmweZtNmZKVKvDG++OYOvAdrkVGOXT7dws/fpq8xYuQu7AfBhcXSj3Rkitbd9uUyZNinxRt1oBrSfvE6JYTFzc3AAo1rI0lwWwzAO9xyRR0/BSeJYqQr0hinsodWnHmru9NgRR5yjVvSFRSnnxF/FBJo3nzFvKhYMliRCddofGoAo6dSvU9PrHFthItXLEcPSYOZ/7gsdyMjLbOVwYDHvkSu+D9ypemULnSnNm1/5FyLP17D/1e/5p+r3/Nju3HafdEYiu0UuVi3LwZS2Tk9VTrvNa3PblyufHtjBWplhUr7k2e3O4cP27/dyjq+GlyFy+CR2E/lIsLxZ5oSXCKzyzhxk2Wt+7B6s4vsrrzi0QePWmt9F1z56LJjMkc/XouEYePp7GVB3f95CncixXFrVBiHt92rQnfnvyZmW/eZGeHbuzp3os93Xtx7fgJjo4cx/VTp4n8Zy+5y5bGkDMnymgkf+0a3Lpw0SG5hMgI6V3ONwtoC6CUag58AgwGapLYon/GkWEsZjPLPv6K176fijIa2L9kNaH+F2k78FUCj5/m5JZdNOrdnbIN62BOMBNz7Tp/jv/EkRGwmM38+NFUxs2dgcFgZPNfywg4d55nB7/J+WMnObB5Gy+MHEJOD3fenj4FgPBgE58PTLwk7P1f5lC4dEncPNz5dvNKZr37IUd27klrk2nSZgt7pnxDu5mfoAwGzi1dS7T/JWoO6EPEiTNc2bqbSr26UahBLXSCmbhr19kx8TMA3D3z0+67T9AWza3QcLa/+6n9OygbZtJmC2s/nkHvmZ9iMBo4/Pdqwv0v0fytVwg+cZqzW3ZTt/dTlGpQG0tCAjHXbrAsabvFalWl8Wu9sSQkoLVmzeSviYm+d0v4QVnMZpZ+/CWvf/85BqOBfUtWYfK/SPuBrxFw/BQntuyi8/D+5PBw58UvEi+Jiw4O5ach4zC6uDBg/gwAYm/c5Pexk7GYzfbtIOCfPadp0LAiv/5vZOLlfFP+tC6bPXcI/V7/Gi/vvLz4cmsuXQpl1pzBAPy9ZDerVu4DoHXrGmzedNjuLJD4mR36dAbNvpuCMhi4uHQN185fovKAPkSdOEPwXQeSKZXp9RS5ixWmcr8XqZx0ymb7gDHERUXbkcfMmc+/pMZXn6MMBoJXrOLWhYuUeuM1rp06RUQa43gSrt/gyu8LqfvjLLTWRO7+J9U4AJH9ZefBeI6mdBrDmJVSh7XWNZKefwuEaa3fT5o+pLWu+QDb0GOrtbQ/qYN8cnQLvSrVzeoYNv44uZ+farbL6hhWrxxan+3yTK7eJqtjWI0/spFR1VqkXzATfXZ0K61bjMnqGFabtk5hUa22WR3D6pmDG9jcMPt8Zq32bM3qCP8FmVoTNxlx0OEDV3Z+XitbHk2k1+I3KqVctNYJQBsSz+8/6LpCCCHEf4IztfjTq7x/B7YqpcKBGGA7QNKP+VzN4GxCCCGEcLD0RvVPVkptBAoB63TyeQEDief6hRBCiP887fDh6tlXut31WutUo1S01o9+rZMQQgiRzThTV7/z3JVACCGEEDJATwghhHCmZrATvVUhhBBCSItfCCGEkMF9QgghhBNxov5vJ3qrQgghhJAWvxBCCOFEzWAneqtCCCGEkBa/EEIIp6ecqBmc5t35HCTDNyCEEOKxk6k/pdd4yiGH11W7xtTMlj8HmCkt/qcq1MqMzTyQv08fpEKJklkdw8bpSxe5/FL2uVVw8V/2s7pe66yOYdVx36ZsdSvlP07up2O5alkdw8bqs0fp2Ht1VsewWv17R1bUzT7foS77N+H/XL2sjmFVZuE+fqjZPqtj2Oh7aF1WRxCZxIk6N4QQQgghFb8QQgjhRGRwnxBCCKfnTIP7nOitCiGEEEJa/EIIIZyeQX6rXwghhHAeBifq/3aityqEEEIIafELIYRwesrgPL81Jy1+IYQQwolIi18IIYTTc6Zz/FLxCyGEcHrOVPE70VsVQgghRLZo8ddq1pjXx4/EYDCw/s+/+WvOjzbLK9etTd9xIyhZoRyfvzOW3Ws3WJe9PGIIdVo0A2Dhd3PYufrRbjTRrEULxr83EYPRyJ9/LGDOzJk2y11z5OCzadOoUq0q0VHRDBs0iMCAAOvyQoULs3LDer758kvmzZ5Djpw5+W3hAnLkyInRxcjaVauZMX36I2Vzq9aIAi+NAIOBm1v+5tqK+TbLczXrQv5eQzFHhQJwff1Cbm5dCkD+noNxq9kUgGt/z+XWP+sfKcPdvBrVo9LwQSiDgYClqzg///d7lvNt1Yzan33Azpf7c+3kmeT35OtDs4U/cm7OfC78utDuPDWaNqLPuBEYDAY2LfqbZXNt91GnPi/Q+plumM1mrkdG8f27kwgPCgFgzOyvKVejGqf/PcRnA4bZnQWgTrMm9H93NAajkTUL/+LP2T/YLK9arw5vjh9FqQrlmTJsFDvWJH4u1RvUo9/4UdZyxUqXYsrbo9i9YZPdmfr3qUS9mt7E3Tbzxcyj+F+8lqrMh2Pq4pk/J0aj4tipKL6bdxyLhlLF8zC4bxXc3FwIDYvhs28Pcysmwa483o3qUWVE4nfo8t+r8L/Pd8ivdTPqfvYB21/qz9WTZ8hfpSLVxr0DgFKKM7PnE7Jlh11ZANxrNMLr1eEog4FrG5cSvdT2O5SnRRcKvjSEhMgwAK6uWcj1TUtxq1IHrz7vWMu5Fi6B6avx3Nq31a48RRrXpeGoARgMBk4vWcORHxfYLK/4TGcq9XwSbbEQfyuGnR9+SfT5yygXI83ee4eCFctiMBo5u2IDR+b9YVcWZ5AdW/xKKU9gAVASuAg8p7WOuke54sBcoBiJd8TtpLW+eL/XzfKK32Aw8ObEMbz36gAiTCamLvqNvZu2EuB/3lomPDiYr8e+x1OvvWyzbp0WTSlduRLDnuqFaw5XPvplLv9u20nMzZsPnWHih5N49YUXMYWEsGjZMjZtWI//2XPWMs/2fI5rV6/SvkVLOnXtyogxYxg2aJB1+ZgJ77J9yxbr9O24OPr0fp5bt27h4uLC/xYtYtuWLRw+ePDhdpAyUKDPaEI/HYg50oTfpJ+59e82EoIu2BS79c96on7+zGaeW40muJasSMj451GurviMm0XM4V3o2IfbP6kYDFQZNZS9g0YSawqj8fyZhG7bxY0Ll2yKGT3cKdmrB9FHT6R6iUrDBhC2a699OZIog4HXJoxmct+BRJhMfLzwZw5s3kagf/I+unjyFOOeXcTt2Dja9erBCyOG8NU74wBYMe8Xcri50bbn0w7JYzAYGPj+eMa90o/wkBC+WvwH/2zazOVzyd/p0KBgvhg9gR59+9ise+SffQx68lkAcufLy7wNq/h3xy67M9Wr6U1hv1z0HbaNimXzM6hvFYZN2J2q3CdfHbJW6OPfrkWzhoXYujuYt/tVZe5vpzl6MpL2LYvSo0spfvnz7KMHMhioOnoo/wwcSYwpjGY/z8R0n+9QqV49iErxHbp27gI7Xu6PNlvIWdCT5r/PwbR9F9psefQ8yoB331EEfTSIhAgTRT+Zz83924gPtP1/dmPXesLnTbWZF3v8AAGjXkh8W7nyUnzGX8Qc3vPoWUj8TjceO4g1/cdw0xTOk7/N4PLW3USfv2wt4796M6cWrQSgeIuGNBj+JmsHjqdUu+YYXV1Z8uybGN1y0uOvOZxfs5kbQSa7MoksMQbYqLWeopQakzQ9+h7lfgYma63XK6VyA2n+Z7jvMY5SapVSqqQdgR9IuepVCb50BVNAIAnxCexYuZYGbVralAkNDObS6bNoi+17KVa2NCf2/4vFbCYuJpZLp89Su3njh85QvWZNLl28RMCVK8THx7Ny+XLatLO9ZWbrdu1ZsngxAGtXraJRk+TttGnfnsArVzh7xvYP4a1btwBwcXHBxdUFrR/+cpEcZaqQYLqCOSwQzAnc2rMOjzotHmhd1yKliTv1L1jM6LhY4q+cw716o4fOcLf8VSpy80ogMYHB6IQEgtdvwqdF6v1evv9rnP/5d8y3b9vM92nRhFtBIdw4f9HuLABlq1ch5PIVQgMCMccnsGvVOuq2tt1HJ/Ye4HZsHABnDx/D09fXuuzYnn3E3rzlkCwA5atXI+jSZUKuBJAQn8DWlatp2KaVTZnQwCAunj6T5neiWYf27N+2g7jYWLszNazjw8btgQCcOhdNbg8XCuTPmarcnUrfaFS4uhis+YoUysXRk5EA/HsknKb1/ezKc+c7dCvpOxS4bhO+9/gOVej/Gv7zf8eS4jtkiYuzVvKGnDngEf5f3S1n2SrEh1whITTx/9mNXevJVe/B/p+llKthG24d3I2+HWdXHu+qFbh2JYjrgSFYEhI4v3YrxVva7p/4FN9ZF3e35N2gNS7ubiijAZecObDEJ3D7huO+348rg8HxDwfoBtzpepoPPHV3AaVUZcBFa70eQGt9Q2ud5geeVrQfgXVKqfFKKddHivwAPH19CA9JPhKNMJnw9PV+oHUvnjpDrWaNyeHmRp4C+anaoC5efg//B8nXz5eQ4CDrtCk4GF8/31RlgoMSy5jNZq5fv06BAgXw8PDgjQH9+ebLr1K9rsFg4O9Vq9j17wF2bd/BkUOHHjqbsYAP5sjk/ZMQGYqxgE+qch71WuM3+Xe8Bn+K0TMxe/zlM7hXb4zKkRND7ny4VaqDsaBvqnUflpu3F7GmUOt0rCkcN2/bzyxvhXK4+XoTtvMf2/fj7kbpl3txbo5tN6o9PH18iEjxHYo0heLpm3of3dGqRzcObbe/FX0/Xn4+hAWHWKfDQ0wU9H34/d68cwe2rFjlkEwFPd0Ij0g+gAiPjMXLM3XFD/DRmLr8/n0bbsUmsOOfxPdxKeAGjeom7tNmDf3wKuhmVx53n7u+Q6HhuPuk/g65+3kTetd3CBIPHFosmEeLP37g6Cdf2tfaB1w8vUmISPH/LMKEi2fqv0O5GrSm6NT/4fvOlHv+X8rTpB03dq61KwuAh48XN0PCrNO3TGHk8imYqlylnl15dvlP1Hv7DfZ89i0AFzZsJyEmlt7r/6Dnmt84+vMibl+7bnemx102rfh9tdbBSc9DgHv9ISkPRCul/lJKHVRKTVVKpfkDxPft6tda/6mUWg1MAPYrpX4hRfeB1nraQ78FBzu0cw9lq1Xh0z9+4mpkFKcPHcFiMWdqhkHD3mb+3B+srfuULBYLT3XqRJ68efl29izKlS/P2TNn7vEq9ok5uJ2bu9dCQjy5Wz1NwTffJ/STAcQe+4ccpavgO3Ee5uvRxJ07Chb7/kA+EKWoOGwARz/4NNWisv1e4eLvizDH2N+KfRRNu3akdNVKfPBSvyzZ/oMq4O1FqQrlOJCBByj38+6U/bi6Ghg1sAY1qhbk4NEIps86yoA+lendvSx7/g0lISGDv0dKUeWdARx6P/V3CCD6+Cm29nyN3CWLU/ODMYTu+gfL7fgMjXTzwHau70z8f5a3bXd8B75H0KS3rMuN+QuSo3hZbh1OfQolo5xcsJyTC5ZTumMrar7xAtsmTMW7agUsFgu/t+9Nzjx56PzjFwTt+ZfrgSHpv6BwKKVUPyDlH5vZWuvZd5XZANyrxTo+5YTWWiul7tW95QI0A2oBl0kcE/AK8MM9ylpXSMtt4CaQE8hDOucN7kj5ZmfNmpVm2UhTKF4pWtcFfX2JNIWlsYatRd//wKLvE9/fO59/TNCFy+mskZopxIRfocLWad9ChTCFmFKVKVS4MKaQEIxGI3ny5CEqKooaNWvyRMdOjBg7lrx582LRFuLi4vht/s/Wda9fu8Y/u3bTrGWLh674zVGh1hY8gIunj3UQ3x2WG1etz29s+Zv8vYZYp68tm8e1ZfMAKDjgI+JDHn7/3C02LBy3FC1qN18vYsOSPzMXDw/ylClF/e8TBzPmLOhJnS8+4sDwd8lfpSJ+rZtTYfCbuObJjbZYMMfd5vKffz9ynsjQUAqm+A55+voQaQpNVa5qo/p0f/M1Pni5HwnxGVdJhIeE4l0o+f+xl58vEaaHO7/avNMT7Fq3CXPCow+g69KuOB1aFwPgzPmrNq10L083wiPv3x0dH29hzwETDev4cvBoBAFBNxn/yT4Aivh5UL/mg/XK3U9M6F3fIR8vYkJTf4cazUr+DtWb9hH73nmXqykGid64eJmEWzHkKVPKZv7DSogMwyVFC96loK91EN8dKf+fXdu4FM8Xh9gsz92oHTf3bgGz/Y2PW6Hh5PJL3scevt7cDI24b/nza7bQZFxinjIdWxO4cx86wUxsVDShh47jVaW8VPzpMGbAL/clVfKz0ynT9n7LlFImpVQhrXWwUqoQkPoPGwQAh7TW55PW+RtoSBoVf1rn+DsAhwAPoLbW+j2t9Qd3Hum8kdla67pa67r9+qXdsjp79DiFShbHp2hhXFxdaNr5CfZu2pLmOtbwBgN58ucDoESFcpSoUI6DOx/+aPvo4cOULFWSosWK4urqSueuXdm03nb0+6YN6+neowcAT3TqxJ5diS2xF559jjZNm9KmaVPmz5vHrG+/5bf5P1PA05M8efMCkDNnTho3a8r5c/4Pne32+RO4+hXD6F0YjC54NGxPzL/bbPdDvuQuQPfazYm/M/BPGTDkTtw/rsXK4lq8HLFH7Rt0BHD1xClyFS+Ce2E/lIsLhdq1JnRb8n5PuHmTje26s7Xb82zt9jzRx05wYPi7XDt5hn/6vW2df/H3xZz/6X92VfoA/kdP4FeiGN5FCmN0daFxp/Yc2Gy7j0pWqsAb749j6sB3uBaZalCsQ505eozCJUvgW7QILq4utOjckT0btzzUa7Ts0tHubv4V6y8zaOxOBo3dye79Jto0KwJAxbL5uXkrgaho24rfLafRet7fYFDUq+VDQNANAPLlzQGAUtCre1lWbbxiV7arJ06Rq1jyd6hI+9aY7voOrWvbnU1PPs+mJxO/Q3cqfffCfihj4p8udz9fcpcsxq0g+yq1OP8TuBYqjkvS/7Pcjdtxc7/td8iYP/n/Wa66zYkPsB34l7tJ+8QeAQcIO36avMWLkLuwHwYXF0o/0YLLW23/tuUtntxYKdasAVcvJ47huBkcSqH6NQFwcXPDu1oloi/Y93mJLLMMuDMCuA+w9B5l9gH5lVJ3jhRbA6lHVKeQVot/PPCs1vr4QwZ9KBazmTmTPuW9ud9hNBrYsHgpV86dp/eQAZw7doJ9m7ZStlplxnwzjdx581K3VXN6D+7PkC7PYHRx4ePfEluzt27c4MuR47E8wtG22Wxm0sSJzP35Z4xGI4sXLuTc2bMMeWcYx44cZdOGDSxasJCp06exbusWrkZHM2zQ4DRf08fHhynTvsBoMKAMBtasWMmWTY9wSZbFTOTPU/EZOQMMRm5uW0Z84HnyPf0mty+cJObgNvI80Qv3Ws3BYsZy4xoRs99PXNfFBd935yS+TMxNImZOAAecCtFmCyc+m0G9rz9FGY0ELFvNjfMXKffmK1w9eYbQbZnbPW0xm/nxo6mMmzsDg8HI5r+WEXDuPM8OfpPzx05yYPM2Xhg5hJwe7rw9fQoA4cEmPh+YeAnW+7/MoXDpkrh5uPPt5pXMevdDjux89AMki9nMzA8+5qN532M0Glm3aAmXz/nz0tCBnDl6nH82baF8tSpM+O4rcufNQ4NWLXhxyFv079QdAJ8ihfHy8+Po3v3275wk+w6GUa+mN/O+bEFsnJnps45Yl33zSRMGjd2Jm5uR90fUxtXVgFKKIyciWLkhscJo2bgQXdqXAGDX3hDWbQm453YelDZbOD51Bg1mJH6HriR9h8onfYdMaXyHPGtWo2yf3lgSEkBrjk75ivirqS9NfCgWM+HzPqPQ+K9RBiPXNi8jPuA8BZ57kzj/k9w6sI18HXuRq25ztDkBy41rhH6X3P5x8S6Ei5cvsSf+tS9HEm22sHvKN3SY+THKYODM0rVE+1+i9oCXCT9xhstb91C5VzcKN6iFJcFM3LXrbJuYeLXBiQXLaD5pBE8vng0ozi5bR9TZC2lvUGTLy/mAKcBCpVRf4BLwHIBSqi7QX2v9utbarJQaAWxUSingADAnrRdVjzLS/CHppyrUyuhtPLC/Tx+kQomSWR3DxulLF7n8Ut2sjmFV/Jf9rK7XOqtjWHXct4lelbLP/vnj5H46lquW1TFsrD57lI69V2d1DKvVv3dkRd3s8x3qsn8T/s/Vy+oYVmUW7uOHmu3TL5iJ+h56tN9AyUAqMzfW7cd/HV4ZLn21dqa+hweVPY9xhBBCCJEhsvwHfIQQQoislk27+jOEE71VIYQQQkiLXwghhNMzOlEz2IneqhBCCCGkxS+EEMLpGbLl+PuMIRW/EEIIpydd/UIIIYR4LEmLXwghhNOTy/mEEEII8ViSFr8QQgin50zn+KXiF0II4fScqeJ3orcqhBBCCGnxCyGEcHrO1OLPlNvyZvQGhBD/b+9+Y+Wo6jCOfx+Kqa1Ng6QgII2NxVhRkipVIiEhYIzWGC0NpBQ1LUmDNmlMGimoJFBAXiBRBCrxD0JpQwqkFpUgjb6wAtJoEEtpLZYixfgv2KoxhYq2/HxxzjXDzd3ZS7p7Zm7n+SSTuztzdue5c/bumXPm7F6zo07Rr9RZtmnw/5b3joXt/Le87vGbmVnn+eN8ZmZmdlRyj9/MzDpvUisH5YfDDb+ZmXVelyb3dehXNTMzMzf8ZmZmHeKG38zMrEN8jd/MzDqvS9f43fCbmVnnHXtMd6b1d+gcx8zMzNzjNzOzzuvSUH+HflUzMzNzj9/MzDrP39xnZmbWIR7qNzMzs6OSe/xmZtZ57vGbmZnZUUkR0XSGcZF0WUR8p+kcVW3L5Dz12pYH2pfJeeo5T39tzGSvNZF6/Jc1HWAMbcvkPPXalgfal8l56jlPf23MZBUTqeE3MzOzI+SG38zMrEMmUsPfxmtGbcvkPPXalgfal8l56jlPf23MZBUTZnKfmZmZHbmJ1OM3MzOzI9Sahl/SAkkhaU5l3QckbZH0rKQnJT0k6Yy8bbWkP0naVlmOG2CeqyTtlLQ9P/dZOcvvKvvbmMveKunqUY/95qCyVJ73cN7vU/l4nJ3Xz8rH7iuVsjMk/VfSmnx/taTLB51pnNkO5m2/lfQtSUVedz3qcIWkPfl4zSiRo0+ee/JraoekOyW9oeE838t1uF3SRknTmsxT2XarpAOlstRlkrRW0vOV94G5DeeRpBsk7Za0S9LnS+Wp5JKkxyTNr6y7SNLm0llsHCKiFQtwH/AocG2+/xZgL3B2pcw5wIJ8ezVw+ZCyfBDYCkzO92cApwBbgHljlJ8O/B54e16eB44bQq4DldsfAX6eb8/K+/9NZftyYBuwZtjHaxzZduTbxwKPAAsLvJ561eF7c6a9wIxh5xhHno8ByssGYHnDeaZXynwd+GKTefLtecD66mus4WO0FriwZJY+eS4F1gHH5PUnls6W9/seYBfwRmAa8Cwwu4ksXuqXVnxlb+5VnAOcBzwIXAOsAO6OiMdHykXEY4UinQzsi4hX8n735ZxjFo6If0m6CliTV10dEf8ccsbpwD8q918GdkmaFxFPAIuA+0lvDKWNzgZARByS9DhwWoEMY9Yh8GfoXZdN5cmZfgWc2nCekSwCpgClJgH1+pubBNwEXAJcUChLv0yFY/TNsxy4JCJezetfbCJcROyQ9CBwJfAmYF1EPNdEFqvXlqH+TwKbI2I3sF/SmcC7gSf7PG5lZbjtZwPM8xNgZh46u13SuZVt91T2edPIyojYALyZ1GNaP8AsVVPyfp8B7gCuH7X9XuBiSTOBw1QalQL6ZUPSVOBDwNMF8tTVYRNq8+Qh/s8ApYZGe+aRdBfwV2AOcFvDeVYAP4qIvxTKMZ5MADfk4fabJU1uOM9sYJGkJyQ9LOkdhfKM5VrSSdp84KsN5rAabWn4F5MaLfLPxaMLSPplvn51S2X1zRExNy/nDSpMRBwAziR9A9XfgPskLc2bP1XZ56pKvlNJZ+SnDPG66MG83znAR4F1em33YzPwYeBi0qWTkuqyzZa0DfgF8FBEPDzsMH3qsLhx5LkdeCQiHm06T0RcShop2kUaOWoqz5eBiyh38jGeTEuBL5FOit4PHE/q4TaZZzLw74iYB3wXuLNEnh4ZXyK996wfGZmw9ml8qF/S8cD5wBmSAphEGl68G3gf8EOAiDhL0oXAx0vkiojDpGv6WyQ9DSzp85BbSJco3pV/rqovfsT5tipNTjuhsu4/kn4NfAE4HfjEMDO8jmzPRcTcBnKMVYdrS+fol0fSNaRj9dk25BnZJule4ArgrobybAD2A3vyOeRUSXsiosSlol6ZlkTE2rz5lTw6MrRJs+PJA/wR2JSLPECh+qrxal6spdrQ47+QdHb4toiYFREzSZPjfgosVZ4dnk0tEUjSO0cNl80FXqgpPx84kTTB5npgoaTTh5xxDukkaf+oTV8DroyIvw9z/3VqspXM8LrqcNh65ZG0jDQZcvHINdoG8/xB0ml5u0gnjs80mOfbEXFSfl+YBbxcstGvqbOT83YBC4AdTeYBfkCaHwVwLrC7RB6buBrv8ZOG9W8cte77ef0i4EZJbwVeBPYB11XKrZT06cr9BRGxdwCZpgG3KX088BCwhzS8tpF0jf9gLrePNALxDdIs3wBekrSKNNHv/AFkqZqSh8whzQJfkntm/y8QETuBnQPe70CyFTZmHSp91OkK4CRgu6QfR8SypvKQrqW/AGzNx2pTRFzX60mGnOdzwAOSppPq8CnSp0NK6HV8mtQr0/2STiAdo22k49ZknkOk96WVwAGgxOvZJjB/c5+ZmVmHtGGo38zMzApxw29mZtYhbvjNzMw6xA2/mZlZh7jhNzMz6xA3/GZmZh3iht/MzKxD3PCbmZl1yP8AEwKMrFPjrL0AAAAASUVORK5CYII=)
%% Cell type:code id: tags:
``` python
X = df.drop(['Y'],axis=1) # drop Y from our dataframe
Y = df['Y'] # create a dataframe with just the Y values
nX =(X-X.mean())/X.std() # create nX, a normalised version of X
feature_names = X.columns
```
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# split into a training set with 80% of the data, and a testing set as the remainder
X_train, X_test, Y_train, Y_test = train_test_split(nX,Y,test_size=0.8)
linear = LinearRegression() # instantatiate the linear regression model
linear.fit(X_train,Y_train) # fit the data to the model
training_score = linear.score(X_train,Y_train) # calculate rsq for the training set
# use the independent variables for the testing set to predict the target variable
preds_linear = linear.predict(X_test)
# calculate the correlation of the predicted and actual target variables
rsquared_linear = r2_score(Y_test,preds_linear)
# print the training and testing scores
print("Training score is",training_score)
print("Testing score is",rsquared_linear)
```
%% Cell type:markdown id: tags:
Again we can plot the linear regression coefficients, but this time we compare them against our original linear regression coefficients to investigate the variability. It can now be seen that the effect of the blood serum measurements can have considerable variability.
%% Cell type:code id: tags:
``` python
# create a new dataframe with the regression coefficients from the normalised data
ncoefs = pd.DataFrame(linear.coef_.transpose(),columns=['Normalised'],index=feature_names)
# add our original coefficient importance to this dataframe
# ncoefs = pd.concat([ncoefs,coefs],axis=1)
# ncoefs.columns =['Normalised','Original'] # change the column names to show the new and original coefficients
# do a similar horizontal plot as before
ax = ncoefs.plot(kind='bar',figsize=(10,7))
plt.title('Linear Regression')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAG2CAYAAADY0keOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAg80lEQVR4nO3df7hcVX3v8fcHEogiAgIiJGBQUSBAAKPQev1RQUWwoq1a/AEE9Kb4oIilKAqtWKWXFpUKqJVeK7alAkWtVNEKilQrqAEhEJRKJUoAMaAB1OpN8Hv/mH1wCCe/OGdlZpL363nmyd577Vn7O5OTfGavvWafVBWSJKmNjQZdgCRJ6zODVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaaS0leXaSmwddx/ogycIkzxt0HVJLBq20EkkWJTlwxe1V9bWqetogalpRklOTLEvy8yRLk3wjye8Muq41VVWzquqrg65DasmglUZEkikrabqwqh4DbANcAfxLg2Mnif9fSI+A/3CktZTkeUkW960vSvKnSRYkuTfJhUmm9bW/JMl1fWece/W1nZTkv5Pcn+SmJC/va5ub5D+TnJnkHuDUVdVVVcuB84HpSbbt+tgiyceS3Jnk9iTvTbJx17ZxkvcnuTvJrUnelKTGAj3JV5OcluQ/gV8CT0qya5LLkvw0yc1JXtVX78Hda7i/O9afdtu3SfK57vX/NMnXxkK7f9QgyaZJ/ibJHd3jb5Js2v+eJzkhyU+613PUI/sblNYtg1aaHK8CDgJ2BvYC5gIk2Qf4e+CPga2BjwKXjAUI8N/As4EtgHcD/5Rk+75+9wN+AGwHnLaqApJsAhwB3AP8rNt8HrAceAqwD/BC4A1d2/8GXgzsDewLvGycbg8H5gGbA0uAy4B/Bh4PHAZ8OMnu3b4fA/64qjYH9gC+0m0/AVgMbNu9jncC49379WRg/66e2cAzgVP62p9A732aDrwe+FCSrVb+jkjDwaCVJsdZVXVHVf0U+Dd6YQG9kPpoVX2zqh6oqk8Av6YXKFTVv3TP+01VXQh8n17AjLmjqs6uquVV9T8rOfarkiwF/odeeL6iqpYn2Q44GDi+qn5RVT8BzqQXkND7cPDBqlpcVT8DTh+n7/OqamF3tnwQsKiqPt7V8x3gU8Aru32XAbsneWxV/ayqru3bvj3wxKpa1l3jHi9oXwv8RVX9pKqW0PvgcXhf+7KufVlVXQr8HBiKa+XSqhi00uT4cd/yL4HHdMtPBE7ohk2XdoG4I7ADQJIj+oaVl9I7E9ymr6/b1uDYF1XVlvTOFm8Ent537KnAnX39f5Te2ShdDf39j3es/m1PBPZb4bW8lt6ZJsAf0gv2Hya5sm9S1hnALcCXkvwgyUkreR07AD/sW/9ht23MPV3gj+l/n6WhtbLJFZImx23AaVX1sGHfJE8E/g44ALiqqh5Ich2Qvt3W+NdrVdXdSeYB85P8c3fsXwPbrBBQY+4EZvSt7zhetyu8liur6gUrOf63gUOTTAXeBFwE7FhV99MbPj4hyR7AV5J8u6q+vEIXd9AL84Xd+k7dNmmkeUYrrdrUJNP6Hmv74fTvgGOS7NfN3N0sySFJNgc2oxdkSwC6yT17TKTYqroZ+HfgbVV1J/Al4P1JHptkoyRPTvLcbveLgLckmZ5kS+Dtq+n+c8BTkxyeZGr3eEaS3ZJskuS1SbaoqmXAfcBvutf1kiRPSRLgXuCBsbYVfBI4Jcm2SbYB/hz4p4m8H9IwMGilVbuU3rXPscepa/PkqppP77rpOfQmKN1CN1Gqqm4C3g9cBdwF7An85yTUfAYwL8nj6U2O2gS4qTv+xfSul0LvQ8CXgAXAd+i91uX0gnC813I/vclUh9E70/wx8FfA2MSuw4FFSe4DjqE3rAywC3A5vWuqVwEfrqorxjnEe4H5XT03ANd226SRFn/xuySAJC8G/raqnjjoWqT1iWe00gYqyaO6775OSTIdeBfwmUHXJa1vPKOVNlBJHg1cCexKb1j888Bbquq+gRYmrWcMWkmSGnLoWJKkhgxaSZIaGqobVmyzzTY1c+bMQZchSdJaueaaa+6uqm3HaxuqoJ05cybz588fdBmSJK2VJD9cWZtDx5IkNWTQSpLUkEErSVJDQ3WNVpK0dpYtW8bixYv51a9+NehSNgjTpk1jxowZTJ06dY2fY9BK0ghbvHgxm2++OTNnzqT3C5LUSlVxzz33sHjxYnbeeec1fp5Dx5I0wn71q1+x9dZbG7LrQBK23nrrtR49MGglacQZsuvOI3mvDVpJ0oQk4YQTTnhw/X3vex+nnnrqOq3hec973oP3YTj44INZunTphPpbtGgRe+yxxyRU5jVaSVqvzDzp85Pa36LTD1ntPptuuimf/vSnecc73sE222yz1sdYvnw5U6ZMXhxdeumlk9bXZPCMVpI0IVOmTGHevHmceeaZD2tbtGgRz3/+89lrr7044IAD+NGPfgTA3LlzOeaYY9hvv/1429vexty5c3njG9/I/vvvz5Oe9CS++tWvcvTRR7Pbbrsxd+7cB/t74xvfyJw5c5g1axbvete7xq1n5syZ3H333fziF7/gkEMOYfbs2eyxxx5ceOGFAFxzzTU897nP5elPfzovetGLuPPOOx/cPnv2bGbPns2HPvShSXt/DFpJ0oQde+yxnH/++dx7770P2f7mN7+ZI488kgULFvDa176W44477sG2xYsX841vfIMPfOADAPzsZz/jqquu4swzz+SlL30pb33rW1m4cCE33HAD1113HQCnnXYa8+fPZ8GCBVx55ZUsWLBgpTV98YtfZIcdduD666/nxhtv5KCDDmLZsmW8+c1v5uKLL+aaa67h6KOP5uSTTwbgqKOO4uyzz+b666+f1PfGoJUkTdhjH/tYjjjiCM4666yHbL/qqqt4zWteA8Dhhx/O17/+9QfbXvnKV7Lxxhs/uP77v//7JGHPPfdku+22Y88992SjjTZi1qxZLFq0CICLLrqIfffdl3322YeFCxdy0003rbSmPffck8suu4y3v/3tfO1rX2OLLbbg5ptv5sYbb+QFL3gBe++9N+9973tZvHgxS5cuZenSpTznOc95sNbJ4jVaSdKkOP7449l333056qij1mj/zTbb7CHrm266KQAbbbTRg8tj68uXL+fWW2/lfe97H9/+9rfZaqutmDt37iq/avPUpz6Va6+9lksvvZRTTjmFAw44gJe//OXMmjWLq6666iH7TnTy1Kp4RitJmhSPe9zjeNWrXsXHPvaxB7f97u/+LhdccAEA559/Ps9+9rMfcf/33Xcfm222GVtssQV33XUXX/jCF1a5/x133MGjH/1oXve613HiiSdy7bXX8rSnPY0lS5Y8GLTLli1j4cKFbLnllmy55ZYPnnGff/75j7jOFXlGK0maNCeccALnnHPOg+tnn302Rx11FGeccQbbbrstH//4xx9x37Nnz2afffZh1113Zccdd+RZz3rWKve/4YYbOPHEE9loo42YOnUqH/nIR9hkk024+OKLOe6447j33ntZvnw5xx9/PLNmzeLjH/84Rx99NEl44Qtf+IjrXFGqatI6m6g5c+aUv49Wa2Oyv8oAa/Z1BmlYfPe732W33XYbdBkblPHe8yTXVNWc8faf8NBxkmlJvpXk+iQLk7y7275zkm8muSXJhUk2meixJEkaNZNxjfbXwPOrajawN3BQkv2BvwLOrKqnAD8DXj8Jx5IkaaRMOGir5+fd6tTuUcDzgYu77Z8AXjbRY0mSNGomZdZxko2TXAf8BLgM+G9gaVUt73ZZDEyfjGNJkh5qmObarO8eyXs9KUFbVQ9U1d7ADOCZwK5r+twk85LMTzJ/yZIlk1GOJG0wpk2bxj333GPYrgNjv4922rRpa/W8Sf16T1UtTXIF8DvAlkmmdGe1M4DbV/Kcc4FzoTfreDLrkaT13YwZM1i8eDGeqKwb06ZNY8aMGWv1nAkHbZJtgWVdyD4KeAG9iVBXAK8ALgCOBD470WNJkh5q6tSp7LzzzoMuQ6swGWe02wOfSLIxvaHoi6rqc0luAi5I8l7gO8DHVtWJJEnrowkHbVUtAPYZZ/sP6F2vlSRpg+W9jiVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGpgy6AEnaUMw86fNN+l10+iFN+tXk8IxWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSG/O09egh/u4hGjT+zGnYTPqNNsmOSK5LclGRhkrd02x+X5LIk3+/+3Gri5UqSNFomY+h4OXBCVe0O7A8cm2R34CTgy1W1C/Dlbl2SpA3KhIO2qu6sqmu75fuB7wLTgUOBT3S7fQJ42USPJUnSqJnUa7RJZgL7AN8EtquqO7umHwPbreQ584B5ADvttNNkliPpEfK6pzR5Jm3WcZLHAJ8Cjq+q+/rbqqqAGu95VXVuVc2pqjnbbrvtZJUjSdJQmJSgTTKVXsieX1Wf7jbflWT7rn174CeTcSxJkkbJZMw6DvAx4LtV9YG+pkuAI7vlI4HPTvRYkiSNmsm4Rvss4HDghiTXddveCZwOXJTk9cAPgVdNwrEkSRopEw7aqvo6kJU0HzDR/iVJGmXeglGSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIb8NXmSpHG1uBXnhngbTs9oJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYmJWiT/H2SnyS5sW/b45JcluT73Z9bTcaxJEkaJZN1RnsecNAK204CvlxVuwBf7tYlSdqgTErQVtV/AD9dYfOhwCe65U8AL5uMY0mSNEpaXqPdrqru7JZ/DGw33k5J5iWZn2T+kiVLGpYjSdK6t04mQ1VVAbWStnOrak5Vzdl2223XRTmSJK0zLYP2riTbA3R//qThsSRJGkotg/YS4Mhu+Ujgsw2PJUnSUJqsr/d8ErgKeFqSxUleD5wOvCDJ94EDu3VJkjYoUyajk6p69UqaDpiM/iVJGlXeGUqSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGpqUWzBKkjRIM0/6/KT3uej0QyalH89oJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKmh5kGb5KAkNye5JclJrY8nSdIwaRq0STYGPgS8GNgdeHWS3VseU5KkYTKlcf/PBG6pqh8AJLkAOBS4abyd77nnHs4777zGJWlVDtrknib9nnfekib9tqi3Va2jZJR+DqzVf18w3LW2DtrpwG1964uB/fp3SDIPmAcwffr0Ne746h+0+YHd/0lbT3qfo1Rriz5bGqV6W/wctHr9o/S+Wms7o1TvMNeaqmrXefIK4KCqekO3fjiwX1W9abz958yZU/Pnz1+jvmee9PlJq7PfotMPmfQ+R6lWtdPi58CfAWk4JLmmquaM19Z6MtTtwI596zO6bZIkbRBaDx1/G9glyc70AvYw4DWT0bGf5CVJo6Bp0FbV8iRvAv4d2Bj4+6pa2PKYkiQNk9ZntFTVpcClrY8jSdIw8s5QkiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1NKGgTfLKJAuT/CbJnBXa3pHkliQ3J3nRxMqUJGk0TZng828E/gD4aP/GJLsDhwGzgB2Ay5M8taoemODxJEkaKRM6o62q71bVzeM0HQpcUFW/rqpbgVuAZ07kWJIkjaJW12inA7f1rS/utj1MknlJ5ieZv2TJkkblSJI0GKsdOk5yOfCEcZpOrqrPTrSAqjoXOBdgzpw5NdH+JEkaJqsN2qo68BH0ezuwY9/6jG6bJEkblFZDx5cAhyXZNMnOwC7AtxodS5KkoTXRr/e8PMli4HeAzyf5d4CqWghcBNwEfBE41hnHkqQN0YS+3lNVnwE+s5K204DTJtK/JEmjzjtDSZLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDUwZdwIZg0emHDLoESdKAeEYrSVJDBq0kSQ1NKGiTnJHke0kWJPlMki372t6R5JYkNyd50YQrlSRpBE30jPYyYI+q2gv4L+AdAEl2Bw4DZgEHAR9OsvEEjyVJ0siZUNBW1Zeqanm3ejUwo1s+FLigqn5dVbcCtwDPnMixJEkaRZN5jfZo4Avd8nTgtr62xd22h0kyL8n8JPOXLFkyieVIkjR4q/16T5LLgSeM03RyVX222+dkYDlw/toWUFXnAucCzJkzp9b2+ZIkDbPVBm1VHbiq9iRzgZcAB1TVWFDeDuzYt9uMbpskSRuUic46Pgh4G/DSqvplX9MlwGFJNk2yM7AL8K2JHEuSpFE00TtDnQNsClyWBODqqjqmqhYmuQi4id6Q8rFV9cAEjyVJ0siZUNBW1VNW0XYacNpE+pckadR5ZyhJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhqaUNAmeU+SBUmuS/KlJDt025PkrCS3dO37Tk65kiSNlome0Z5RVXtV1d7A54A/77a/GNile8wDPjLB40iSNJImFLRVdV/f6mZAdcuHAv9QPVcDWybZfiLHkiRpFE2ZaAdJTgOOAO4Ffq/bPB24rW+3xd22Oyd6PEmSRslqz2iTXJ7kxnEehwJU1clVtSNwPvCmtS0gybwk85PMX7Jkydq/AkmShthqz2ir6sA17Ot84FLgXcDtwI59bTO6beP1fy5wLsCcOXNqvH0kSRpVE511vEvf6qHA97rlS4AjutnH+wP3VpXDxpKkDc5Er9GenuRpwG+AHwLHdNsvBQ4GbgF+CRw1weNIkjSSJhS0VfWHK9lewLET6VuSpPWBd4aSJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKmhSQnaJCckqSTbdOtJclaSW5IsSLLvZBxHkqRRM+GgTbIj8ELgR32bXwzs0j3mAR+Z6HEkSRpFk3FGeybwNqD6th0K/EP1XA1smWT7STiWJEkjZUJBm+RQ4Paqun6FpunAbX3ri7ttkiRtUKasbocklwNPGKfpZOCd9IaNH7Ek8+gNL7PTTjtNpCtJkobOaoO2qg4cb3uSPYGdgeuTAMwArk3yTOB2YMe+3Wd028br/1zgXIA5c+bUePtIkjSqHvHQcVXdUFWPr6qZVTWT3vDwvlX1Y+AS4Ihu9vH+wL1VdefklCxJ0uhY7RntI3QpcDBwC/BL4KhGx5EkaahNWtB2Z7VjywUcO1l9S5I0qrwzlCRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDbX6NXmSVrDo9EMGXYKkAfCMVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqaFU1aBreFCSJcAPG3S9DXB3g35bsNY2RqlWGK16rbUNa22nRb1PrKptx2sYqqBtJcn8qpoz6DrWhLW2MUq1wmjVa61tWGs767peh44lSWrIoJUkqaENJWjPHXQBa8Fa2xilWmG06rXWNqy1nXVa7wZxjVaSpEHZUM5oJUkaCINWkqSGDFpJkhpar4I2yWNX0bbTuqxFkiRYz4IW+OrYQpIvr9D2r+u0kjWQ5Owkm4+zfdcklw+ippVJ8ieregy6vhUleVmSP03yokHX8kglGaqZnEk2TvLHSd6T5FkrtJ0yqLpWJsmjk7wtyYlJpiWZm+SSJH+d5DGDrm91kvzXoGsYT5K9+panJjmle1//MsmjB1nbipK8Kck23fJTkvxHkqVJvplkz3VVx/oWtOlbftwq2obFj4HrkrwGHvyP4a+BfwM+NNDKHm7z1TyGRpIPA28Ftgbek+TPBlzSSiV53EoeWwMHD7q+FXwUeC5wD3BWkg/0tf3BYEpapfOA7YCdgc8Dc4Az6P1f8JHBlfVwSe5Pcl/3uD/J/cCTx7YPur4VnNe3fDrwFOD9wKOAvx1EQavwxqoau9XiB4Ezq2pL4O2sw1rXq6/3JLm2qvZdcXm89WGR5EnA2fTCagfgIuC9VfXLgRY2wpLcCMyuqge6T9hfq6qnD7qu8SR5gN79vfs/CFa3Pr2qNhlIYeNIsqCq9uqWpwAfpnfP2FcDV1fVPoOsb0VJrquqvZMEuBPYvqqqW79+7LUMgyRnAVsCJ1bVXd22W6tq54EWNo4k3xn7u05yHfCMqlo2pO/rzVX1tG7521X1jL62Beuq1inr4iDr0OO7Ycz0LdOtj3uz5yHwm+7PKcDGwHeHMWS7/whWqqqOW1e1rIH/V1UPAFTVL7v/AIbVD4ADqupHKzYkuW0A9azKg6FfVcuBeUneBXwFGNqh2C5cL63urKJbH6ozjKo6LsnTgU8m+VfgHHofuIbRFkleTm9EdNOqWgbD+b4CFyc5D/gL4DNJjgc+AzwfeNi/uVbWt6D9O347jNm/DPB/1305q9YNaR4JnFxVFyaZDnwwyRvoDXncNNgKH+IY4EZ6Z9x3MJxD8WN2TbKgWw69IbgF3XIN0ydu4G+ArRj/H/1fr9tSVmt+koOq6otjG6rq3UluZ8iGYjvzkzymqn5eVUePbUzyZOD+AdY1rqq6JsmBwJuAK4FpAy5pZa4EXtotX51ku6q6K8kTGLLf4FNVJyeZC3wSeDKwKTCP3pyd166rOtaroK2qdw+6hrW0DbBPVd0PUFW3A69I8mLgU8BugyxuBdsDrwT+CFgOXAhcXFVLB1nUSgzT+7Y63wLuGltJcgTwh/SGk08dUE0r80Fg8djKCrVuN6iiVuGj9M60fw4PqfdHwMsGV9bDJXkGcFtV/Zje9e/NgHcn+SDw7qr66WArfIgPA4ur6k7ova9Jxn4OXjnQylbQva9frKrzuvUj6f0MPIbeCOK6qWM9u0Y7C3hyVV3SrZ8JbNE1n1NV1w6suLWUZNOq+vWg6xhPkhnAYcCfAG+vqn8ccEmr1c08vKeG7Ac+ybXAgVX10yTPAS4A3gzsDexWVa8YZH39RqlWGK16rbWNYal1fZt1fDoPHbp4Eb3ZhlcAfz6QilYhyUV9y3+1QvO/reNy1kiSfYG3AK8DvgBcM9iKHi7J/km+muTTSfbpJkfdCNyV5KBB17eCjfvOVv4IOLeqPlVVf0ZvNucwGaVaYbTqtdY2hqLW9S1ot6+qb/St39e9qf9Ib5h22OzSt/yCFdqGavJWkr9Icg29s9grgTlV9fohu4485hzgL+ldl/kK8IaqegLwHOD/DLKwcWzczeAFOIBevWOG7dLOKNUKo1WvtbYxFLUO25syUQ/5PmdV7d+3+vh1XMuaWNUw5lANcQKnALcCs7vHX3aTeYdxgtGUqvoS9D4gVNXVAFX1vSGcgPxJ4MokdwP/A3wNel+uB+4dZGHjGKVaYbTqtdY2hqLW9S1o70iyX1V9s39jkv3pzZQdNo9Osg+9kYVHdcOy0AuvRw2urHEN3ff5VuE3fcv/s0LbUH2AqarT0ruL2fbAl/quIW9E71rS0BilWmG06rXWNoal1vVtMtQz6c2GPQ8Ym/j0dHpfofmjqvrWgEobV5Ir+O3NCeChIZCq+r11X9WaG+IJRg8Av+C3H1jGvpccYFpVTR1UbZI2POtV0AIk2Q44FphFL7gWApcDh1XVsYOsbUXdB4Pb+qbJj009XwScOkxT+rtRgdOBnwLvAcaue28EHNH/3UpJ0m+td0E7phuGfTW973XdCnyqqs4ZbFUPNSxTz9dEkvnAO+l9Xepc4MVVdXWSXYFP1pDdfk+ShsV6dY02yVPpheur6X3N50KGewh23KnnwKe6e4gOk1GaYCRJQ2N9+3rP9+jdw/IlVfW/qups4IEB17QqQzH1fA2NzAQjSRomw/af+UT9Ab07Fl2R5Iv0hmKH+XRrKKaer6HZ6f26rtCbIT32q7vC8N6TVZIGbr28RtvdJ/RQekPIzwf+AfjM2NDnMOkmGY1NPf9Ft+2pwGNG6ZaRkqTxrZdB2y/JVnQ3w6+qAwZdjyRpw7LeB60kSYO0vk2GkiRpqBi0kiQ1ZNBKktSQQStJUkMGrSRJDf1/xjZbWO2T1+YAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
To investigate the variability we can use the `sklearn` methods `cross_validate` and `RepeatedKFold`. The first of these performs a number of runs of a model. The second splits the data in n sections and repeats the calculations m times. This gives n.m runs to investigate the variability of the coefficients. The variability of these can then be plotted using a boxplot.
We can see `AGE`, `SEX`, `BP`, `BMI` and `S6` have very low variance, whereas `S1`-`S5` have high variance due to overfitting. Of the low variance features, `AGE` and `S6` seem to have little effect on the predictions.
%% Cell type:code id: tags:
``` python
from sklearn.model_selection import cross_validate, RepeatedKFold # import sklearn methods
rng = np.random.RandomState(1) # make sure the results are repeatable
# cross_validate takes the particular model, in this case linear regression which we instantatiated earlier,
# and undertakes a number of runs according the method specified by cv=
# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times, giving 25 runs
# return_estimator=True returns the fitting data for each run
scores = cross_validate(linear, nX, Y, cv=RepeatedKFold(n_splits=5, n_repeats=5), return_estimator=True)
# take the results for each simulation (estimator), extract the coefficients for each run
# and add them to a dataframe with columns being the feature names
coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)
# plot the descriptive statics of the coefficients in a box and whisker plot to show variability
ax = coefs.plot(kind='box',figsize=(10,7))
plt.title('Linear Regression')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAGrCAYAAABnmlbLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAiuElEQVR4nO3de5RkZXnv8e/j0MKoMMYMeAFkjDdam6CmvSSZHO1oCF6WkByjTIyCpw3BFTvGpXFGKytAjr0yxGW8jPEoSRMgMS1EoyGiEU1aYxtvjYgMtBgig4AYBtQRZfD0wHP+qN1zasbunm6q3t5VNd/PWrXo2nv33k8XNfWr/e53v29kJpIkqYwH1F2AJEn9zKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1ZaoYj4lYi4vu46+kFEXBsRz6m7Dqkkg1ZaRETsiIjn7b88Mz+XmU+so6b9RcQ5ETEXET+KiB9ExH9ExC/WXddyZeaTM/MzddchlWTQSj0iIg5ZZNUlmfkQYD0wBfxDgWNHRPh5Id0P/sORViginhMRt7Q83xERb4yIr0fEroi4JCIOa1n/ooj4WssZ58+3rNsSEf8VEXdFxHUR8Rst686IiM9HxDsi4k7gnKXqysw9wAeAoyPiyGof6yJiIiJui4hbI+KtEbGmWrcmIt4eEXdExI0R8dqIyPlAj4jPRMR4RHweuBv4uYg4PiI+FRHfi4jrI+KlLfW+oPob7qqO9cZq+fqI+Fj1938vIj43H9qtrQYRcWhEvDMivlM93hkRh7a+5hHxhoi4vfp7XnX//g9Kq8uglTrjpcDJwGOAnwfOAIiIpwIXAL8H/CzwfuCy+QAB/gv4FWAdcC7wdxHxyJb9PhP4FvBwYHypAiLigcArgTuB71eLLwT2AI8DngqcBLy6Wve7wPOBpwBPA05dYLevAM4EDgd2Ap8C/h44CjgNeG9EPKnadgL4vcw8HBgC/q1a/gbgFuDI6u94C7DQ2K8N4FlVPScCzwD+uGX9I2i+TkcDo8BfRsTPLP6KSN3BoJU6492Z+Z3M/B7wzzTDApoh9f7M/FJm3puZFwE/oRkoZOY/VL93X2ZeAvwnzYCZ953M3JaZezJz9yLHfmlE/ADYTTM8X5KZeyLi4cALgD/MzB9n5u3AO2gGJDS/HLwrM2/JzO8DWxfY94WZeW11tnwysCMz/6aq5yrgw8BvVdvOAU+KiCMy8/uZ+dWW5Y8EjsvMueoa90JB+3LgTzPz9szcSfOLxyta1s9V6+cy8+PAj4CuuFYuLcWglTrjuy0/3w08pPr5OOANVbPpD6pAPBZ4FEBEvLKlWfkHNM8E17fs6+ZlHPvSzHwozbPF7cAvtBx7ALitZf/vp3k2SlVD6/4XOlbrsuOAZ+73t7yc5pkmwP+kGew3RcRnWzplvQ24AbgiIr4VEVsW+TseBdzU8vymatm8O6vAn9f6Oktda7HOFZI642ZgPDN/qtk3Io4D/gp4LvCFzLw3Ir4GRMtmy55eKzPviIgzgZmI+Pvq2D8B1u8XUPNuA45peX7sQrvd72/5bGb+2iLH/wpwSkQMAK8FLgWOzcy7aDYfvyEihoB/i4ivZOa/7reL79AM82ur54+ulkk9zTNaaWkDEXFYy2OlX07/CjgrIp5Z9dx9cES8MCIOBx5MM8h2AlSde4baKTYzrwc+CbwpM28DrgDeHhFHRMQDIuKxEfHsavNLgddFxNER8VBg8wF2/zHgCRHxiogYqB5Pj4jBiHhgRLw8ItZl5hzwQ+C+6u96UUQ8LiIC2AXcO79uP5PAH0fEkRGxHvgT4O/aeT2kbmDQSkv7OM1rn/OPc1byy5k5Q/O66XtodlC6gaqjVGZeB7wd+ALw38AJwOc7UPPbgDMj4iianaMeCFxXHf9DNK+XQvNLwBXA14GraP6te2gG4UJ/y100O1OdRvNM87vAecB8x65XADsi4ofAWTSblQEeD3ya5jXVLwDvzcypBQ7xVmCmquca4KvVMqmnhRO/SwKIiOcD78vM4+quReonntFKB6mIWFvd+3pIRBwNnA18pO66pH7jGa10kIqIBwGfBY6n2Sx+OfC6zPxhrYVJfcaglSSpIJuOJUkqqKvuo12/fn1u2LCh7jIkSVqRK6+88o7MPHKhdV0VtBs2bGBmZqbuMiRJWpGIuGmxdTYdS5JUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFdRVE79Lkrrbhi2Xr2j7HVtfWKiS3mHQSpKWbaHg3LDlcgN1CTYdS5JUkEErSVJBBq0kSQUZtJIkFWTQSpJUkL2OJS2Lt3VI949BK2lZFgtOb+2QlmbTsSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQV1LGgjYg1EXFVRHysev6YiPhSRNwQEZdExAM7dSxJknpFJ89oXwfMtjw/D3hHZj4O+D4w2sFjSZLUEzoStBFxDPBC4K+r5wH8KvChapOLgFM7cSxJknpJp85o3wm8Cbivev6zwA8yc0/1/Bbg6IV+MSLOjIiZiJjZuXNnh8qRJKk7tB20EfEi4PbMvPL+/H5mnp+Zw5k5fOSRR7ZbjiRJXaUTYx3/MvDiiHgBcBhwBPAu4KERcUh1VnsMcGsHjiVJUk9pO2gz883AmwEi4jnAGzPz5RHxD8BLgA8CpwP/1O6xJEmr58Rzr2DX7rllbbvc2Z3WrR3g6rNPaqesnlNy9p7NwAcj4q3AVcBEwWNJkjps1+65js/MtNLpFvtBR4M2Mz8DfKb6+VvAMzq5f0mSeo0jQ0mSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFlRywQpLUww4f3MIJF23p8D6hOdnbwcOglSQt6K7ZrY4M1QE2HUuSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQd7eI0laVKdvx1m3dqCj++sFBq0kaUHLvYd2w5bLO36/bT+x6ViSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyLGOJf2UE8+9gl2755a9/XIHnl+3doCrzz7p/pYl9SSDVtJP2bV7rsgg8Z2eCUbqBTYdS5JUkEErSVJBBq0kSQV5jVbLstJra04CLUlNBq2WZaHg3LDlcgNVkg7ApmNJkgoyaCVJKsiglSSpIINWkqSC7AylfTj0nqTlmpycZHx8nJuum2XoY4M0Gg02bdpUd1ldx6DVPhx6T9JyTE5O0mg0mJiY4IyP/5BtLziC0dFRAMN2PwatJGnZ5r80f2diCw973lm86pN3E2sO4VWfvJt7nvm7nP4HW3jz1Ufs3d5bAA1aSdIKzAfnmrfdws0Xv5GBgYG96+bmTuKww/7YcN2PQat9HD64hRMu2lJgvwD+45P6xeDgINPT04yMjOxdNj09zeDgYI1VdSeDVvu4a3ar12glHVCj0WB0dJSJiQk2btzI9PQ0o6OjjI+P111a1zFoJUkrNt/haWxsjNnZWQYHBxkfH7cj1AIiM+uuYa/h4eGcmZmpu4yDWqkzT2/v6S0nXHRCsX1fc/o1xfYt1SUirszM4YXWeUarfayk2dhJBfqXlxCkznFkKEmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKDVik1OTjI0NMRNf/5ihoaGmJycrLskSepabd9HGxHHAhcDDwcSOD8z3xURDwMuATYAO4CXZub32z2e6jF//+OPr/ssP/j3i/nZ57+ORz//Sdxxy3W88jWv5w8mr+LBT3r23u29v1aSmjoxYMUe4A2Z+dWIOBy4MiI+BZwB/Gtmbo2ILcAWYHMHjqcazAfn0NBmLv3IZMtA4qcwNfUsxsbG2H7Zn9dXoCR1qbabjjPztsz8avXzXcAscDRwCnBRtdlFwKntHkv1m52dZePGjfss27hxI7OzszVVJEndraPXaCNiA/BU4EvAwzPztmrVd2k2LS/0O2dGxExEzOzcubOT5aiA+amxWjk1liQtrmNBGxEPAT4M/GFm/rB1XTZnLlhw9oLMPD8zhzNz+Mgjj+xUOSpkfmqsqakp5ubmmJqaYnR0lEajUXdpktSVOjKpQEQM0AzZD2TmP1aL/zsiHpmZt0XEI4HbO3Es1cupsSRpZTrR6ziACWA2M/+iZdVlwOnA1uq//9TusdQdNm3aZLBK0jJ14oz2l4FXANdExNeqZW+hGbCXRsQocBPw0g4cS5KkntJ20GbmNBCLrH5uu/uXVmqlc556z+/CSswdu27tQMf3KXW7aPZT6g7Dw8M5MzNTdxnqQ05SX46vrQQRcWVmDi+0ziEYJUkqyKCVJKkgg1aSpIIMWkmSCurIgBVSXU489wp27Z5b1rbL7UW7bu0AV599UjtlSdJeBq162q7dcx3v8VrithZJBy+bjiVJKsgzWvW0wwe3cMJFWzq8TwDvC5XUGQatetpds1ttOpbU1Ww6liSpIINWkqSCbDpWz+t0U68D30vqJINWPW2512cd+F5SXWw6liSpIM9oJalmzqHc3wxaSarZQsHp5Y7+YdBK6ksrOUs00FSSQau+s9gH7GLL/ZDtT54lqlsYtOo7fpBK6ib2OpYkqSCDVpKkggxaSZIKMmglSSrIzlCStEpOPPcKdu2eW/b2y71Fad3aAa4++6T7W5YKM2glaZXs2j1XpFe8cyh3N5uOJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyF7HknpaL90yc/jgFk64aEtH99ncL4BjfHcrg1ZST+ulW2bumt3aM7Wqc2w6liSpIINWkqSCDFpJkgryGq0kraIS11PXrR3o+D7VOQatJK2SlXSE2rDl8iIdp7T6DFpJPc1bZtTtDFpJPc1bZtTtDFpJUl9a6ZelUk31Bq0kqS8tFpyrff3b23skSSrIM1pJy7JUM9xC6+wxKzX1ddBOTk4yPj7O7Owsg4ODNBoNNm3aVHdZUk8yOKX7p2+DdnJykkajwcTEBBs3bmR6eprR0VEAw1aStGr6NmjHx8eZmJhgZGQEgJGRESYmJhgbGzNoJanPlJgusVNTJfZt0M7OzrJx48Z9lm3cuJHZ2dmaKpIklVJiusRO3Uvdt0E7ODjI9PT03jNagOnpaQYHB2usSpJUQokRwjo1OljfBm2j0WB0dPSnrtGOj4/XXZqkDnOgfpUYIcwz2gOYvw47Nja2t9fx+Pi412elPuNA/ep2fRu00Axbg1VSt1vszGmx5X5ZWFinWzY61arRV0HbLeNaStJK+FnUvm5u2eiroO2WcS0lSZrnWMeSJBVUPGgj4uSIuD4iboiIzs/OLElSFyvadBwRa4C/BH4NuAX4SkRclpnXlTyuJEndMhFG6Wu0zwBuyMxvAUTEB4FTgAWD9s477+TCCy/seBEnP/BOLrxwZ8f3Kx3Mbr/9dm666SbuvvtuHvSgB3Hcccdx1FFH1V3WkvwsOLicc/zKti+RP1A+aI8Gbm55fgvwzNYNIuJM4EyAo48+etk7nrnp++y5975lb//Fb925rO0OWfMAho/7mWXvVzoY3X777dx444088YlPZN26dezatYvrr78eoOvDVlptkZnldh7xEuDkzHx19fwVwDMz87ULbT88PJwzMzPL2nepnsT2UJYObGhoiG3btu0zxOnU1BRjY2Ns3769xsqW5r9vlRIRV2bm8ELrSneGuhU4tuX5MdUyST3MSTuk5SvddPwV4PER8RiaAXsa8Nud2HGJAaSb+4VODCIt9bPBwUHOPfdcPvrRj+4d4vTUU0910g5pAUWDNjP3RMRrgU8Ca4ALMvPaTuy7xADSUGZwcqnfjIyMcN5553Heeedx1lln8b73vY/Nmzdz1lln1V2a1HWK30ebmR/PzCdk5mMz06lzpD4wNTXF5s2bueCCCzj88MO54IIL2Lx5M1NTU3WXJnWdnh6C0amxpHrMzs5y1VVX8da3vnXvsrm5Of7sz/6sxqqk7tSzQdvNA0hL/W5wcJDp6el9eh1PT097jVZagGMdS1qxRqPB6OgoU1NTzM3NMTU1xejoKI1Go+7SpK7Ts2e0kuozP8/z2NjY3l7H4+Pjzv8sLaCvgrZbxrWUDgabNm0yWKVl6KugNTglSd3Ga7RSF5mcnGRoaIg1a9YwNDTE5ORk3SVJalNfndFKvWxycpJGo8HExAQbN25kenqa0dFRAJtopR7mGa3UJcbHx5mYmGBkZISBgQFGRkaYmJhgfNxxXqReZtBKXcKB+qX+ZNBKXWJ+EIhWDgIh9T6v0UpdYn4QiP2v0dp0fP8sdruft/pptRm0UpdwEIjOMjzVLSIz665hr+Hh4ZyZmam7DEmSViQirszM4YXWeY1WUt/z/mTVyaZjSX3N+5NVN5uOJfW1oaEhtm3bts+UflNTU4yNjbF9+/YaK1M/Warp2KCV1NfWrFnDPffcw8DAwN5lc3NzHHbYYdx77701VqZ+4jVaSQct709W3QxaSX3NSepVNztDSepr3p+sunmNVpKkNnmNVuoR3u8p9R+bjmu02FisC3E4uf7n/Z5Sf7LpuMts2HK5oXqQ8n5PqXfZdCz1AOejlfqTQSt1Ce/3lPqTQSt1Ce/3lPqTnaFWwYnnXsGu3XPL3n65naTWrR3g6rNPur9lqct4v2c5k5OTjI+P731dG42Gr6tWjUG7CnbtnivSwWklvZbVGzZt2mQAdJi9uVU3m44l9bXx8XEmJiYYGRlhYGCAkZERJiYmGB8fr7s0HSS8vWcVnHDRCcX2fc3p1xTbt9QPnL1Hq2Gp23tsOl4Fd81utelYqsl8b+7W+5Ptza3VZNOxpL5mb27VzTPaVVLi7HPd2oEDbyQd5OzNrbp5jbbLOASjJPUeh2CUJKkmBq0kSQV5jbZGi123XWi5zcmS1JsM2hoZnpLU/2w6liSpIINW6iKTk5MMDQ2xZs0ahoaGmJycrLskSW2y6VjqEg5+L/Un76OVusTQ0BDbtm3bZ6jAqakpxsbG2L59e42VSTqQpe6jNWilLuHg91LvcsAKqQfMD37fysHvpd7nNVqpSzQaDV72spfx4Ac/mG9/+9s8+tGP5sc//jHvete76i5NUhs8o5W6UDdd0pHUHoNW6hLj4+Nccskl3Hjjjdx3333ceOONXHLJJYyPj9ddmqQ22BlK6hJ2hpJ6l52hpB5gZyipPxm0UpdoNBqMjo4yNTXF3NwcU1NTjI6O0mg06i5NUhvsdSx1ifnRn8bGxpidnWVwcJDx8XFHhZJ6nNdoJUlqk9doJUmqSVtBGxFvi4hvRMTXI+IjEfHQlnVvjogbIuL6iPj1tiuVJKkHtXtG+ylgKDN/Hvgm8GaAiHgScBrwZOBk4L0RsabNY0mS1HPaCtrMvCIz91RPvwgcU/18CvDBzPxJZt4I3AA8o51jSZLUizp5jfZ/AZ+ofj4auLll3S3Vsp8SEWdGxExEzOzcubOD5UiSVL8D3t4TEZ8GHrHAqkZm/lO1TQPYA3xgpQVk5vnA+dDsdbzS35ckqZsdMGgz83lLrY+IM4AXAc/N/3+v0K3AsS2bHVMtkyTpoNJur+OTgTcBL87Mu1tWXQacFhGHRsRjgMcDX27nWJIk9aJ2R4Z6D3Ao8KmIAPhiZp6VmddGxKXAdTSblH8/Mx0VXZJ00GkraDPzcUusGwec30uSdFBzZChJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpoHYnFZDUhg1bLl/R9ju2vrBQJZJKMWilGi0WnBu2XG6oSn3CpmNJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDqm7AOlgceK5V7Br99yyt9+w5fIDbrNu7QBXn31SO2VJKsyglVbJrt1z7Nj6wo7uczlhLKleNh1LklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQV1JGgjYg3RERGxPrqeUTEuyPihoj4ekQ8rRPHkSSp17QdtBFxLHAS8O2Wxc8HHl89zgT+T7vHkSSpF3XijPYdwJuAbFl2CnBxNn0ReGhEPLIDx5Ikqae0FbQRcQpwa2Zevd+qo4GbW57fUi1baB9nRsRMRMzs3LmznXIkSeo6B5z4PSI+DTxigVUN4C00m43vt8w8HzgfYHh4OA+wudSzDh/cwgkXbenwPgE6O5m8pM46YNBm5vMWWh4RJwCPAa6OCIBjgK9GxDOAW4FjWzY/plomHbTumt3Kjq2dDcUNWy7v6P4kdd79bjrOzGsy86jM3JCZG2g2Dz8tM78LXAa8sup9/CxgV2be1pmSJUnqHQc8o72fPg68ALgBuBt4VaHjSJLU1ToWtNVZ7fzPCfx+p/YtSVKvcmQoSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKKjVNnqQFdHqi9nVrBzq6P0mdZ9BKq2TH1hcue9sNWy5f0faSupdNx5IkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkLf3SDVa6r7ahdZ5y4/UewxaqUYGp9T/bDqWJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqqO2gjYixiPhGRFwbEX/esvzNEXFDRFwfEb/e7nEkSepFh7TzyxExApwCnJiZP4mIo6rlTwJOA54MPAr4dEQ8ITPvbbdgSZJ6SbtntK8BtmbmTwAy8/Zq+SnABzPzJ5l5I3AD8Iw2jyVJUs9pN2ifAPxKRHwpIj4bEU+vlh8N3Nyy3S3Vsp8SEWdGxExEzOzcubPNciRJ6i4HbDqOiE8Dj1hgVaP6/YcBzwKeDlwaET+3kgIy83zgfIDh4eFcye9KktTtDhi0mfm8xdZFxGuAf8zMBL4cEfcB64FbgWNbNj2mWiZJ0kGl3abjjwIjABHxBOCBwB3AZcBpEXFoRDwGeDzw5TaPJUlSz2mr1zFwAXBBRGwH/i9wenV2e21EXApcB+wBft8ex5Kkg1FbZ7SZ+X8z83cycygzn5aZ/9aybjwzH5uZT8zMT7RfqtT/JicnGRoaYs2aNQwNDTE5OVl3SZLa1O4ZraQOmZycpNFoMDExwcaNG5menmZ0dBSATZs21VydpPsrmi293WF4eDhnZmbqLkOqxdDQENu2bWNkZGTvsqmpKcbGxti+fXuNlUk6kIi4MjOHF1xn0ErdYc2aNdxzzz0MDAzsXTY3N8dhhx3GvffaxUHqZksFrZMKSF1icHCQ6enpfZZNT08zODhYU0WSOsGglbpEo9FgdHSUqakp5ubmmJqaYnR0lEajUXdpktpgZyipS8x3eBobG2N2dpbBwUHGx8ftCCX1OK/RSpLUJq/RSpJUE4NWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKqirJhWIiJ3ATQV2vR64o8B+S7DWMnqpVuiteq21DGstp0S9x2XmkQut6KqgLSUiZhabVaHbWGsZvVQr9Fa91lqGtZaz2vXadCxJUkEGrSRJBR0sQXt+3QWsgLWW0Uu1Qm/Va61lWGs5q1rvQXGNVpKkuhwsZ7SSJNXCoJUkqaC+CNqIODUiMiKOb1n2jIj4TET8Z0R8NSIuj4gTqnXnRMStEfG1lsdDV6nWRkRcGxFfr477zKrO61tq+VC17bsj4k/2+92/XI06q+PdW9VzdfUa/lK1fEP1er+1Zdv1ETEXEe+pnp8TEW9crVpXUPfuat11EfG+iKj938Ai74nXRsQN1eu8vu4a5y1S6weq9+/2iLggIgbqrhMWrXWiel98PSI+FBEPqbvOeQvV27Lu3RHxozrra7XIa3thRNzY8jn2lLrrhEVrjYgYj4hvRsRsRPxB0SIys+cfwCXA54Bzq+cPB3YAv9SyzUbg1Ornc4A31lDnLwJfAA6tnq8HHgV8BhheYPsjgG8BP1c9bgQeuor1/qjl518HPlv9vKGq66qW9a8Bvga8p87XeBl1b69+PgT4d+A366hxGe+Jp1b17gDW11njMmp9ARDVYxJ4TRfXekTLNn8BbKm71qXqrX4eBv629X3djbUCFwIvqbu+Zdb6KuBi4AHV8qNK1nEIPa76RroRGAH+GTgbeC1wUWb+x/x2mTldT4X7eCRwR2b+BCAz7wCIiAU3zswfRkQDeE+16E8y8werUOdCjgC+3/L8bmA2IoYzcwZ4GXApzTdxN9m/bgAyc09E/AfwuNUvaR8LvieA78Di742aLFkrQER8GTimhtr2t1itAETzhV0LdEtv0MU+G9YAbwN+G/iN+srbx4o+x2q2WK2vAX47M++rlt9esojam8064BTgXzLzm8CdEfELwJOBrx7g917f0sQxVbzKpiuAY6vmivdGxLNb1n2gpZ63zS/MzEngZ2h+E//bVapz3tqqnm8Afw387/3WfxA4LSKOBe6l5QO3Zgeqm4h4EPBc4JrVLm4/S70nus2StVZNxq8A/qWW6va1aK0R8TfAd4HjgW11Fbifxep9LXBZZt5WY237W+p9MF410b4jIg6tq8AWi9X6WOBlETETEZ+IiMeXLKIfgnYTzQ98qv9u2n+DiPhS1Q7/rpbF78jMp1SPkdUoNDN/BPwCcCawE7gkIs6oVr+8pZ4/aqn9GJrfyh5Vw/Wk3VU9xwMnAxfHvl9b/wX4NeA0ms333WKpuh8bEV8DPg9cnpmfqKtIOOB7oqsso9b3Av+emZ+robx9LFVrZr6KZsvLLM2WmNotUu9bgN+ie74MAEu+tm+m+eXl6cDDgM111ThviVoPBe7J5jCMfwVcULqQnn3Q/J95N82JCHYANwPfpnkGc+5+274EuLD6+Rxqun64QE3/zCLXaKttPgycDmwF3rbK9f1ov+f/DRzFvtc6L6B5dvAw4Ay67BrtYnV362P+PdHyfAddco12qVppXrL5KNU1r2577P+6Vsv+B/Cxumtbot656t/WjupxH3BD3bUt87V9Tje+ti2fud8AHlMtC2BXyeP2+hntS4C/zczjMnNDZh5Ls8PQp4AzouptWnlQLRW2iIgn7tdE8RSWmK0oIp5PMyAupvnl4Tcj4klFi1y8luOBNcCd+616O7A5M7+3+lUd2BJ1d4WVvifqtFitEfFqmp3ONmV1zatui9T67Yh4XLU+gBfT/MCt3SL1vj8zH1F9tm0A7s7MuvsULPU+eGS1PoBTge2rX92+lvj39VGa/XoAng18s2Qdvd4ZahNw3n7LPlwtfxlwXkQcDdxOc0qkP23Z7vUR8Tstz0/NzB0FawV4CLAtmrcS7QFuoNmk8SGa12h3V9vdAbwIeCfNXnwJ/Dgi/ohmx6hfLVznvLVVMys0v/Wdnpn3trYeZ+a1wLWrVM9yHbDuLrLge6K63eBNwCOAr0fExzPz1fWVCSz+/v0uzQ+vL1Sv8T9m5p8utpNVslCtZwEfiYgjaL4vrqbZW74bLPbadqPFar00Io6k+dp+jebrXbfFat1D8zP39cCPgKL/thyCUZKkgnq96ViSpK5m0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQf8POrt6g3nR79QAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
### Regularization
We now investigate regularization techniques for Linear Regression, to reduce the variance of the model.
#### Ridge regularization
To use Ridge regularization (which adds a penalty term which is proportional to the sum of the squares of the coefficients), we need to find the optimal value of tuning parameter alpha. Generally this would be done using `RidgeCV`, however here we will graphically compare the training and testing scores. What we want to determine is the value of alpha for which we obtain the maximum value of the testing score. To generate the figure we create an array of alpha values, which in this case are logarithmically distributed, and perform a Ridge regularization for each, and store the testing and training scores ( $R^2$ ). Then we plot these against alpha. From the figure we see the optimal value occurs at alpha approximately 20.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# create an array of 21 alpha values logarithmically distributed between 10**(-2) and 10**2
alfas = np.logspace(-2,2,num=21)
# create two arrays for storage of the same size as alfas, but filled with zeros
ridge_training_score = np.zeros_like(alfas)
ridge_rsquared = np.zeros_like(alfas)
# loop over the values in the alfas array, at each loop the current value is alfa
# and idx is incremented by 1, starting at 0
for idx, alfa in enumerate(alfas):
ridge = Ridge(alpha=alfa) # instantatiate Ridge regularization with the current alfa
ridge.fit(X_train,Y_train) # train the model to our data set
# calculate the training score and store in the array ridge_training_score
ridge_training_score[idx] = ridge.score(X_train,Y_train)
preds_linear = ridge.predict(X_test) # calculate the model prediction for the test data
# calculate the correlation between the predicted and actual test data and store in the array ridge_rsquared
ridge_rsquared[idx] = r2_score(Y_test,preds_linear)
plt.scatter(alfas,ridge_training_score,label='Training score') # plot the training score against alpha
plt.scatter(alfas,ridge_rsquared,label='Testing score') # plot the testing score against alpha
plt.xscale('log') # make the x-axis a logarithmic scale
plt.gca().set_xlim(left=.01, right=100); # fix the x-axis limits
plt.title('Ridge regularization')
plt.xlabel('alpha');
plt.legend(loc='best');
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkoklEQVR4nO3df3xU5Zn38c9lQIiSEgv4uARa8FkMBQxBIoiWBaUK7VZB/PFgqeLWrlKrWHfFQrtVarXq0mrFqixd0VrtKlJAWuziKmX9AYpBLBggC1K2JFiltESQoCFezx8ziUOYJDOZM5kzme/79eKVzDn3fc49t/Fc5/5x7mPujoiI5LZjMl0AERHJPAUDERFRMBAREQUDERFBwUBERFAwEBERFAwkzcxsvpl9r4X9bmZ/255lCpqZXWlmL6eQ/ztm9u9Blil63BbrXiRWp0wXQLKbme0E/g9QDxwA/hO4zt0PALj79MyVLju4+w9TPYaZXQl83d0/H3Nc1b0kTC0DCcL57t4NKAWGAbMzW5wjWUQo/9bNTDdkEgqh/B9EspO7/wlYSSQoAGBmj5rZ7TGfZ5rZO2a228y+FpvfzHqY2a/N7H0ze93Mbo/tfjGzgWb2X2b2FzOrNLNLmyuLma02szvM7BXgIHByS/lbOreZ9Yt2Z3VqcvyvN3Pu+8xsV/RY681sdMy+OWa22MweN7P3gSuj2x6P7v+pmR2I+XfYzOZE980ys7fNbL+ZbTazC6PbPwfMB0ZF8+xrpu7/0cy2R7//cjPrHbPPzWy6mW0zs31m9oCZWXP1Kx2PgoEExsz6AF8EtjezfwJwE3AuMAD4QpMkDwAfACcB06L/GvIeD/wX8EvgRGAK8KCZDWqhSJcDVwMFwJ5W8jd77jZ4nUhA/HT0fE+bWdeY/ROBxUAh8ERsRne/zt27RVtanwf+CjwT3f02MBroDnwfeNzM/sbdtwDTgbXRvIVNC2Rm5wB3ApcCfwP8L/Bkk2RfBk4HSqLpxrfhu0uWUjCQICwzs/3ALuA94NZm0l0KPOLub7n7B8Cchh1mlgdcBNzq7gfdfTPw85i8XwZ2uvsj7n7Y3TcAvwIuaaFcj7p7hbsfBiY0lz+BcyfF3R93973R8/wY6AIUxyRZ6+7L3P1jd6+Ndwwz6wUsA66PlhV3f9rdd0fzPQVsA0YkWKypwEJ3f8PdPyTSlTfKzPrFpLnL3fe5+x+B3xHTwpOOT8FAgjDJ3QuAscBAoGcz6XoTCRgN/jfm915EJjTE7o/9/bPAyGgXxr5oV8hUInfyzUk0f2vnToqZ3WRmW8ysJnqe7hxZJy0e28w6E2k5/NLdn4zZfoWZvRlT/iE0X9dN9SamvqMD/HuBopg0f4r5/SDQLcFjSwegwSsJjLv/t5k9CvwImBQnyTtA35jPn4n5fQ9wGOgD/E90W2zaXcB/u/u5yRQpkfzRlkFL5/4g+vM44P3o73GDUHR84GZgHFDh7h+b2V+B2P731pYKvj96nn+JOe5ngZ9Fj7vW3evN7M2Y47Z2zN1EAmLD8Y4HegDVreSTHKGWgQTtJ8C5ZjY0zr5FRAZMB5nZccR0J7l7PbAEmGNmx5nZQOCKmLy/AU4xs8vNrHP03+nRwdNENJu/tXO7+x4iF82vmlledOD7/zZzngIigWUP0MnMbgE+lWAZMbNrgDHAVHf/OGbX8UQu+Hui6f6BSMugwbtAHzM7tplD/wfwD2ZWamZdgB8Cr7n7zkTLJh2bgoEEKnrhfAy4Jc6+3xIJFquIDDKvapLkOiJdKn8CfkHkAvZhNO9+4DwiA7+7o2nuJtIfn0i5Wsvf7Lmj/hGYSaRrZTCwpplTrSTyrMX/EOmWOURyXU6XAScDu2NmFH0nOo7xY2AtkQv/qcArMflWARXAn8zsz3G+//PA94iMk7xDJJhNSaJc0sGZXm4jYWVmdwMnuXsqM3uy7twimaCWgYRG9DmAEosYAVwFLO3o5xYJAw0gS5gUEOme6U2kK+THfDLHviOfWyTj1E0kIiLqJhIREQUDEREhhGMGPXv29H79+mW6GCIiWWX9+vV/dvdebc0fumDQr18/ysvLM10MEZGsYmb/23qq5qmbSEREFAxERETBQEREUDAQERFCOIC8qbqGs+5axczxxUwaVtR6hhjLNlQzd2Ulu/fV0rswP6uPISLSnkIXDACq99Uye8kmgIQvoss2VDN7ySZq6+o7xDHCEJDCcgwRSb/QdhPV1tUzd2VlwunnrqxsvABn8zEagkn1vlqcT4LJsg2Jv4Okox3jrLtW0X/WCs66a1VSeUUkcaENBgC798V9PWxSabPtGGEISGE5RhDBpOE4CigiLQt1MOhdmJ9y2mw7RhgCUliOEURAUkARSUxog0F+5zxmji9OOP3M8cXkd87L+mOEISCF5RhBBKQwBRSRMAtlMCgqzOfOyacmNdA4aVgRd04+laLCfCyLjxGGgBSWYwQRkMIUUNSykDAL3WyiU4u688qsc9qUd9KwopRnqmT6GA35UpmB01GOMXN88REzsyD5gNS7MJ/qOBf+9gwoQcwwE0m30L3cpqyszLVQnTRIdWpq0wsxRAJKMq21s+5aFTegFBXmJ3Tjkmr+BpqmKy0xs/XuXtbW/KFrGYjESrWlFkQLJ9UWShBdVWpdSLopGEiHl+mAEkRXVUvjFsm2lNS6kHgUDEQSkEpACWLsQ60LSbdQziYS6UiCmKUWxMyqIGZFScelloFIO0i1qyosrQvpuBQMRLJAEAPhQYxdgMYdOioFA5EsEYbWhcYdOi6NGYjkiCDGLjTu0HGpZSCSQ1JtXWjcoeNSy0BEEhbErCYJJwUDEUlYEAsYSjipm0hEEhbErCYJJwUDEUlKquMOmpoaTgoGItJuNDU1vDRmICLtRlNTwyuhYGBmE8ys0sy2m9msFtJdZGZuZmXRz53N7OdmtsnMtpjZ7KAKLiLZR1NTw6vVYGBmecADwBeBQcBlZjYoTroC4AbgtZjNlwBd3P1UYDhwjZn1C6DcIpKFNDU1vBJpGYwAtrv7Dnf/CHgSmBgn3Q+Au4FDMdscON7MOgH5wEfA+6kVWUSylaamhlciwaAI2BXzuSq6rZGZnQb0dfcVTfIuBj4A3gH+CPzI3f/S9ARmdrWZlZtZ+Z49e5Ipv4hkkSCWxJD0SHk2kZkdA9wDXBln9wigHugNnAC8ZGbPu/uO2ETuvgBYAJF3IKdaJhEJr1Snpkp6JBIMqoG+MZ/7RLc1KACGAKvNDOAkYLmZXQB8BfhPd68D3jOzV4Ay4IhgICIimZVIN9HrwAAz629mxwJTgOUNO929xt17uns/d+8HvApc4O7lRLqGzgEws+OBM4CtAX8HERFJUastA3c/bGbXASuBPGChu1eY2W1AubsvbyH7A8AjZlYBGPCIu28MouAikrv0FHPwzD1cXfRlZWVeXl6e6WKISEg1fYoZIjOScn0g2szWu3tZW/PrCWQRySp6ijk9FAxEJKvoKeb0UDAQkayip5jTQ8FARLKKnmJODy1hLSJZRS/YSQ8FAxHJOnqKOXjqJhIREQUDERFRMBARERQMREQEBQMREUHBQEREUDAQEREUDEREBAUDERFBwUBERFAwEBERtDaRiOQovTrzSAoGIpJzmr46s3pfLbOXbALI2YCgbiIRyTl6debRFAxEJOfo1ZlHUzAQkZyjV2ceTcFARHKOXp15NA0gi0jO0aszj6ZgICI5Sa/OPJK6iURERMFAREQUDEREBAUDERFBwUBERFAwEBERFAxERAQFAxERQcFARERQMBARERIMBmY2wcwqzWy7mc1qId1FZuZmVhazrcTM1ppZhZltMrOuQRRcRESC0+raRGaWBzwAnAtUAa+b2XJ339wkXQFwA/BazLZOwOPA5e7+ezPrAdQFWH4REQlAIi2DEcB2d9/h7h8BTwIT46T7AXA3cChm23nARnf/PYC773X3+jh5RUQkgxIJBkXArpjPVdFtjczsNKCvu69okvcUwM1spZm9YWY3xzuBmV1tZuVmVr5nz54kii8iIkFIeQDZzI4B7gH+Oc7uTsDnganRnxea2bimidx9gbuXuXtZr169Ui2SiIgkKZFgUA30jfncJ7qtQQEwBFhtZjuBM4Dl0UHkKuBFd/+zux8EngVOC6LgIiISnESCwevAADPrb2bHAlOA5Q073b3G3Xu6ez937we8Clzg7uXASuBUMzsuOpg8Bth89ClERCSTWg0G7n4YuI7IhX0LsMjdK8zsNjO7oJW8fyXShfQ68CbwRpxxBRERyTBz90yX4QhlZWVeXl6e6WKIiGQVM1vv7mWtp4xPTyCLiIiCgYiIKBiIiAgKBiIiQgJrE4mISHzLNlQzd2Ulu/fV0rswn5nji5k0rKj1jCGkYCAi0gbLNlQze8kmausiy61V76tl9pJNAFkZENRNJCLSBnNXVjYGgga1dfXMXVmZoRKlRsFARKQNdu+rTWp72CkYiIi0Qe/C/KS2h52CgYhIG8wcX0x+57wjtuV3zmPm+OIMlSg1GkAWEWmDhkFizSYSEclxk4YVZe3Fvyl1E4mIiIKBiIgoGIiICAoGIiKCgoGIiKBgICIiKBiISK7auAjuHQJzCiM/Ny7KdIkySs8ZiEju2bgIfj0D6qLrCNXsinwGKLk0c+XKILUMRCT3vHDbJ4GgQV1tZHuOUjAQkdxTU5Xc9hygYCAiuad7n+S25wAFAxHJPqkO/o67BTo3WWq6c35ke47SALKIZJcgBn8b0r1wW6RrqHufSCDI0cFjUDAQkWzT0uBvMhfzkktz+uLflLqJRCS7aPA3LRQMRCS7hGnwtwM9uKZgICLZJSyDvw1jFzW7AP9k7CJLA4KCgYhkl5JL4fx50L0vYJGf589r//7/DvbgmgaQRST7hGHwt4ONXahlICLSFmEauwiAgoGISFuEZewiIAoGIiJtEZaxi4BozEBEpK3CMHYRkIRaBmY2wcwqzWy7mc1qId1FZuZmVtZk+2fM7ICZ3ZRqgUVEJHitBgMzywMeAL4IDAIuM7NBcdIVADcAr8U5zD3Ab1Mrqoh0CB3oQa2OJJGWwQhgu7vvcPePgCeBiXHS/QC4GzgUu9HMJgF/ACpSK6qIZL0O9qBWR5JIMCgCdsV8ropua2RmpwF93X1Fk+3dgG8D30+xnCLSEXSwB7U6kpRnE5nZMUS6gf45zu45wL3ufqCVY1xtZuVmVr5nz55UiyQiYdXBHtTqSBKZTVQN9I353Ce6rUEBMARYbWYAJwHLzewCYCRwsZn9K1AIfGxmh9z9p7EncPcFwAKAsrIyb9tXEZHQ694n2kUUZ7tkVCItg9eBAWbW38yOBaYAyxt2unuNu/d0937u3g94FbjA3cvdfXTM9p8AP2waCEQkh3SwB7U6klaDgbsfBq4DVgJbgEXuXmFmt0Xv/kVEEtPBHtTqSMw9XL0yZWVlXl5enuliiIhkFTNb7+5lraeMT8tRiIiIgoGIiCgYiIgICgYiIoKCgYiIoGAgIiIoGIiICHq5jYgka+OiyMJyNVWRZSTG3aKHxtpo2YZq5q6sZPe+WnoX5jNzfDGThhW1njENFAxEJHENS1A3rDzasAQ1KCAkadmGamYv2URtXT0A1ftqmb1kE0BGAoK6iUQkcVqCOjBzV1Y2BoIGtXX1zF1ZmZHyKBiISOK0BHVgdu+rTWp7uikYiEjimltqWktQJ613YX5S29NNwUBEEqclqAMzc3wx+Z3zjtiW3zmPmeOLM1IeDSCLSOIaBok1myhlk4YVUbTrN/R9Yy4n+h7es17sOm0mpw+bkJHyKBiISHJKLtXFPwgbF3H6pluBWjA4iT2ctOlW6HdCRupX3UQiIpkQsplZCgYiIpkQsplZCgYiIpkQsplZCgYiIpkQsplZCgYiIplQcimcPw+69wUs8vP8eRkbnNdsIhGRTAnRzCy1DERERMFAJKdsXAT3DoE5hZGfGxdlukQSEuomEskVWn5aWqCWgUiuCNlDThIuCgYiuSJkDzlJuCgYiOSKkD3kJOGiYCCSK0L2kJOEi4KBSK4I2UNOEi6aTSSSS0L0kJOEi1oGIiKiYCAiIgoGIiKCgoGIiKBgICIiKBiIiAgJBgMzm2BmlWa23cxmtZDuIjNzMyuLfj7XzNab2aboz3OCKrhIztGKo5JGrT5nYGZ5wAPAuUAV8LqZLXf3zU3SFQA3AK/FbP4zcL677zazIcBKoCiowovkDK04KmmWSMtgBLDd3Xe4+0fAk8DEOOl+ANwNHGrY4O4b3H139GMFkG9mXVIss0ju0YqjkmaJBIMiYFfM5yqa3N2b2WlAX3df0cJxLgLecPcPm+4ws6vNrNzMyvfs2ZNAkURyjFYclTRLeQDZzI4B7gH+uYU0g4m0Gq6Jt9/dF7h7mbuX9erVK9UiiXQ8WnFU0iyRYFAN9I353Ce6rUEBMARYbWY7gTOA5TGDyH2ApcAV7v52EIUWyTlacVTSLJFg8DowwMz6m9mxwBRgecNOd69x957u3s/d+wGvAhe4e7mZFQIrgFnu/krwxRfJEVpxVNKs1dlE7n7YzK4jMhMoD1jo7hVmdhtQ7u7LW8h+HfC3wC1m1nALc567v5dqwUVyjlYclTQyd890GY5QVlbm5eXlmS6GiEhWMbP17l7W1vx6AllERBQMREREwUCkfWgpCQk5vfZSJN20lIRkAbUMRNJNS0lIFlAwEEk3LSUhWUDBQCTdtJSEZAGNGYik27hbjhwzgFAvJVFXV0dVVRWHDh1qPbG0u65du9KnTx86d+4c6HEVDETSrWGQ+IXbIl1D3ftEAkFIB4+rqqooKCigX79+mFmmiyMx3J29e/dSVVVF//79Az22goFIe8iipSQOHTqkQBBSZkaPHj1Ix1L/GjMQSUSOPSegQBBe6fpvo5aBSGv0nIDkALUMRFqj5wTa1d69eyktLaW0tJSTTjqJoqKixs8fffRRi3nLy8uZMWNGq+c488wzgypuh6GWgUhr9JxAi5ZtqGbuykp276uld2E+M8cXM2lYUesZm9GjRw/efPNNAObMmUO3bt246aabGvcfPnyYTp3iX7rKysooK2t94c41a9a0uXzp1NJ3Sze1DERao+cEmrVsQzWzl2yiel8tDlTvq2X2kk0s21Ddat5kXHnllUyfPp2RI0dy8803s27dOkaNGsWwYcM488wzqaysBGD16tV8+ctfBiKB5Gtf+xpjx47l5JNPZt68eY3H69atW2P6sWPHcvHFFzNw4ECmTp1Kw7L+zz77LAMHDmT48OHMmDGj8bixKioqGDFiBKWlpZSUlLBt2zYAHnvsMUpKShg6dCiXX345ADt37uScc86hpKSEcePG8cc//jHud3v77beZMGECw4cPZ/To0WzdujXQumyOWgYircmy5wTa09yVldTW1R+xrbaunrkrK1NqHcRTVVXFmjVryMvL4/333+ell16iU6dOPP/883znO9/hV7/61VF5tm7dyu9+9zv2799PcXEx3/jGN46an79hwwYqKiro3bs3Z511Fq+88gplZWVcc801vPjii/Tv35/LLrssbpnmz5/PDTfcwNSpU/noo4+or6+noqKC22+/nTVr1tCzZ0/+8pe/AHD99dczbdo0pk2bxsKFC5kxYwbLli076ruNGzeO+fPnM2DAAF577TWuvfZaVq1aFWhdxqNgINKaLHtOoD3t3leb1PZUXHLJJeTl5QFQU1PDtGnT2LZtG2ZGXV1d3Dx///d/T5cuXejSpQsnnngi7777Ln36HNmiGzFiROO20tJSdu7cSbdu3Tj55JMb5/JfdtllLFiw4Kjjjxo1ijvuuIOqqiomT57MgAEDWLVqFZdccgk9e/YE4NOf/jQAa9euZcmSJQBcfvnl3HzzzUd9twMHDrBmzRouueSSxn0ffvhhm+orWQoG0vFtXJT6hTyLnhNoT70L86mOc+HvXZgf+LmOP/74xt+/973vcfbZZ7N06VJ27tzJ2LFj4+bp0qVL4+95eXkcPny4TWma85WvfIWRI0eyYsUKvvSlL/Fv//ZvCeeN1fDdPv74YwoLCxvHTNqTxgykY2uYFlqzC/BPpoV28OcE2svM8cXkd847Ylt+5zxmji9O63lramooKop0Qz366KOBH7+4uJgdO3awc+dOAJ566qm46Xbs2MHJJ5/MjBkzmDhxIhs3buScc87h6aefZu/evQCN3URnnnkmTz75JABPPPEEo0ePPup4n/rUp+jfvz9PP/00EHni+Pe//33QXy8uBQPp2DQtNK0mDSvizsmnUlSYjwFFhfncOfnUwMcLmrr55puZPXs2w4YNS+pOPlH5+fk8+OCDjQO5BQUFdO/e/ah0ixYtYsiQIZSWlvLWW29xxRVXMHjwYL773e8yZswYhg4dyj/90z8BcP/99/PII49QUlLCL37xC+677764537iiSd4+OGHGTp0KIMHD+aZZ54J/PvFYw0j52FRVlbm5eXlmS6GdBRzCoF4f+MGc/a1b1myxJYtW/jc5z6X6WJk3IEDB+jWrRvuzje/+U0GDBjAjTfemOliAfH/G5nZendvfV5tM9QykHBLdRkITQuVNvrZz35GaWkpgwcPpqamhmuuuSbTRUorDSBLeAWxDISmhUob3XjjjaFpCbQHtQwkvILo7y+5FM6fB937Ahb5ef48zQySjiPaeh7+N8cMT+UwahlIeAW1DISmhUpH1bT1nAK1DCS9UunzV3+/SMvitZ7bSMFA0ifVOf7jbon078dSf7/IJwJcLFHBQJqX6kyeVPv81d+fk1JZwhoii8/Frko6f/58HnvssXQWOXMCbCVrzEDiC2ImTxB9/urvD78glvuI0doS1q1ZvXo13bp1a3xnwfTp09tclnRyd9ydY45J4Z583C0cfuZ6OtUfSrk8ahl0VJm+qwf1+eeCdlruY/369YwZM4bhw4czfvx43nnnHQDmzZvHoEGDKCkpYcqUKezcuZP58+dz7733UlpayksvvcScOXP40Y9+BMDYsWP59re/zYgRIzjllFN46aWXADh48CCXXnopgwYN4sILL2TkyJHEe/h11qxZjedrCFDvvvsuF154IUOHDmXo0KGNrZJ77rmHIUOGMGTIEH7yk58AkWWsi4uLueKKKxgyZAi7du1i7ty5nH766ZSUlHDrrbcmVS/L6s9iVt3Xqfq4J5Da6zDVMgijVO+0wnJXrzn+HV9LNw0Btejcneuvv55nnnmGXr168dRTT/Hd736XhQsXctddd/GHP/yBLl26sG/fPgoLC5k+ffoRrYkXXnjhiOMdPnyYdevW8eyzz/L973+f559/ngcffJATTjiBzZs389Zbb1FaWnpUOfbu3cvSpUvZunUrZsa+ffsAmDFjBmPGjGHp0qXU19dz4MAB1q9fzyOPPMJrr72GuzNy5EjGjBnDCSecwLZt2/j5z3/OGWecwXPPPce2bdtYt24d7s4FF1zAiy++yN/93d8lVDdzV1ZS/dGZLOZM3vFvpVLNahmkRSp35UHcaYXlrl59/h1fO7wF7sMPP+Stt97i3HPPpbS0lNtvv52qqsjxS0pKmDp1Ko8//njCbwibPHkyAMOHD29ciO7ll19mypQpAAwZMoSSkpKj8nXv3p2uXbty1VVXsWTJEo477jgAVq1axTe+8Q0gsupp9+7defnll7nwwgs5/vjj6datG5MnT25shXz2s5/ljDPOAOC5557jueeeY9iwYZx22mls3bq18QU5iQhyqfDwtQx2vxm5gLal3zGIvstM35UHcacVprt69fl3bN37RG9c4mwPiLszePBg1q5de9S+FStW8OKLL/LrX/+aO+64g02bNrV6vIYlq5NdrrpTp06sW7eOF154gcWLF/PTn/60TS+diV2K292ZPXt2m5e6aG4J8bYIYcugjXfDQdxRh+GuPIgLue7qpb20w/TfLl26sGfPnsZgUFdXR0VFBR9//DG7du3i7LPP5u6776ampoYDBw5QUFDA/v37kzrHWWedxaJFkf/PN2/eHDeoHDhwgJqaGr70pS9x7733Ni4tPW7cOB566CEA6uvrqampYfTo0SxbtoyDBw/ywQcfsHTp0rhLVo8fP56FCxdy4MABAKqrq3nvvfcSLne8JcTbKnwtgwbJ3g0HcUcdhrvyIO60dFcv7aUd3gJ3zDHHsHjxYmbMmEFNTQ2HDx/mW9/6Fqeccgpf/epXqampwd2ZMWMGhYWFnH/++Vx88cU888wz3H///Qmd49prr2XatGkMGjSIgQMHMnjw4KOWrN6/fz8TJ07k0KFDuDv33HMPAPfddx9XX301Dz/8MHl5eTz00EOMGjWKK6+8khEjRgDw9a9/nWHDhjV2SzU477zz2LJlC6NGjQIi72Z+/PHHOfHEExMqd8NS4XNXVvJOQjmal9AS1mY2AbgPyAP+3d3vaibdRcBi4HR3L49umw1cBdQDM9x9ZUvnKuud5+VXd2s4YuLLDAexVHEQx7h3SDMX875w41ut54/3eHnn/OTvygOe7ie5IxeXsK6vr6euro6uXbvy9ttv84UvfIHKykqOPfbYTBctrnQsYd1qy8DM8oAHgHOBKuB1M1vu7pubpCsAbgBei9k2CJgCDAZ6A8+b2SnufuQbtJuTzN1wEHfUYbgrD+pOS3f1Igk7ePAgZ599NnV1dbg7Dz74YGgDQbok0k00Atju7jsAzOxJYCKwuUm6HwB3AzNjtk0EnnT3D4E/mNn26PGOHglqKtlujSC6RoI4RhAXc13IRdpVQUFB3OcKckkiwaAIiL1drgJGxiYws9OAvu6+wsxmNsn7apO8R70Pz8yuBq4G+Ez36GBlWy6gkPpFONVjNBxHF3PJYu6OWWoPMUl6pOvtlCkPIJvZMcA9wJVtPYa7LwAWQOS1l9zYxggdxEVYF3LJcV27dmXv3r306NFDASFk3J29e/fStWvXwI+dSDCoBvrGfO4T3dagABgCrI7+4ZwELDezCxLIKyIh06dPH6qqqtizZ0+miyJxdO3alT59gl/SJZFg8DowwMz6E7mQTwG+0rDT3WuAng2fzWw1cJO7l5tZLfBLM7uHyADyAGBdcMUXkaB17tyZ/v37Z7oY0s5aDQbuftjMrgNWEplautDdK8zsNqDc3Ze3kLfCzBYRGWw+DHwz4ZlEIiLSbhJ6zqA9lZWVea6P6ouIJCvV5wxCuByFiIi0t9C1DMxsP1DZDqfqDtSkOV9raZvbn8z2ptuafu4J/LnVkqamrXWZTN5E0qk+g6vPlvYnUm+JbGuPumyuHEHny0R9Nv1c7O4FrRe1GQ1v2wnLPyLjEO1xngXpztda2ub2J7O96bY4n9Nen22ty2TyJpJO9Rlcfba0P5F6S2Sb/l9PrT6D/tvM5W6iX7dDvtbSNrc/me1Nt7X1e6UilXMmmjeRdKrP4Oqzpf2J1Fsy29ItzP+vN7ev3f82w9hNVO4pDILIkVSfwVJ9Bkd1GaxU6zOMLYMFmS5AB6P6DJbqMziqy2ClVJ+haxmIiEj7C2PLQERE2pmCgYiIKBiIiEiWBQMzm2RmPzOzp8zsvEyXJ9uZ2clm9rCZLc50WbKRmR1vZj+P/k1OzXR5sp3+HoOV7PWy3YKBmS00s/fM7K0m2yeYWaWZbTezWS0dw92Xufs/AtOB/5fO8oZdQPW5w92vSm9Js0uS9ToZWBz9m7yg3QubBZKpT/09ti7J+kzqetmeLYNHgQmxG2Ler/xFYBBwmZkNMrNTzew3Tf6dGJP1X6L5ctmjBFef8olHSbBeibyfo+EtgFqNN75HSbw+pXWPknx9JnS9TPlNZ4ly9xfNrF+TzXHfr+zudwJfbnoMi7w95y7gt+7+RpqLHGpB1KccLZl6JfIa1z7Am2RZl2t7SbI+m75XXZpIpj7NbAtJXC8z/Qcc7/3KR70jOcb1wBeAi81sejoLlqWSqk8z62Fm84FhZjY73YXLYs3V6xLgIjN7iMwss5Ct4tan/h7brLm/z6Sul+3WMgiCu88D5mW6HB2Fu+8l0p8obeDuHwD/kOlydBT6ewxWstfLTLcM9I7kYKk+00P1GizVZ7ACqc9MB4PG9yub2bFE3q/c7Gs0pVWqz/RQvQZL9RmsQOqzPaeW/gewFig2syozu8rdDwMN71feAixy94r2KlM2U32mh+o1WKrPYKWzPrVQnYiIZLybSEREQkDBQEREFAxERETBQEREUDAQEREUDEREBAUDkSOY2U4z65lqGpFso2AgIiIKBpK7zGyZma03swozu7rJvn5mttXMnjCzLWa22MyOi0lyvZm9YWabzGxgNM8IM1trZhvMbI2ZFbfrFxJJgYKB5LKvuftwoAyYYWY9muwvBh50988B7wPXxuz7s7ufBjwE3BTdthUY7e7DgFuAH6a19CIBUjCQXDbDzH4PvEpk1ccBTfbvcvdXor8/Dnw+Zt+S6M/1QL/o792Bp6OvJLwXGJyOQoukg4KB5CQzG0vkxR+j3H0osAHo2iRZ04W7Yj9/GP1ZzyfvBfkB8Dt3HwKcH+d4IqGlYCC5qjvwV3c/GO3zPyNOms+Y2ajo718BXk7gmA3ryF8ZSClF2omCgeSq/wQ6xbwn9tU4aSqBb0bTnEBkfKAl/wrcaWYbyLK3CIpoCWuROKIvHf9NtMtHpMNTy0BERNQyEBERtQxERAQFAxERQcFARERQMBARERQMREQEBQMREQH+P9k/qJlwddLYAAAAAElFTkSuQmCC)
%% Cell type:markdown id: tags:
Now can investigate Ridge regularization using the optimal value of alpha approximately 20. Now we see significantly reduced variance in the coefficient and that the most important variables are `BMI`, `BP` and `S5`.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# cross_validate takes the particular model, in this case ridge regularization
# and undertakes a number of runs according the method specified by cv=
# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times, giving 25 runs
# return_estimator=True returns the fitting data for each run
scores = cross_validate(Ridge(alpha=20.), nX, Y, cv=RepeatedKFold(n_splits=5, n_repeats=5), return_estimator=True)
# take the results for each simulation (estimator), extract the coefficients for each run
# and add them to a dataframe with columns being the feature names
coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)
# plot the descriptive statics of the coefficients in a box and whisker plot to show variability
ax = coefs.plot(kind='box',figsize=(10,7))
plt.title('Ridge regression, alpha = 20.')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAGrCAYAAABnmlbLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhnElEQVR4nO3de7xcZX3v8c9XzaGxYNASERSMF5SoCJ5utReOlaqIl9Z7S2w9aGNRj9DLy/YYm9MC2rTxtN6qtYoFQY+NUhFFoha1qEWtNVhAMCpYQxVBgpcIEm2Q3/ljVnDY7NnZYc+zZ2bn83695pWZtdY8z29PZu/vrGc9s1aqCkmS1MadRl2AJEmLmUErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0mRpK3JPnTWdZXkgcuZE3jKMnBSW5McucR1/GJJC8c9rbSpDFoNTaSbEmyvQuJa5OckWTvneur6sVV9apR1jgJquo/q2rvqvrJqGuZFEn2SnJakquS3JDk4iRPmrbN45J8OclNSS5Ict9R1avJYtBq3PxaVe0NHAE8AnjFaMu5rfQM7fcmyV2G1Zbm5S7AN4BfAZYB/wc4K8kKgCT7Ae8D/hS4B7AJeM9IKtXEMWg1lqrqWuCf6AUuAN0e7p/3Pf7jJNck+VaS3+l/fpKfS/LBJD9I8vkkf57kwr71hyb5aJLvJvlKkt8YVEs3rLkuyaeBm4D7z/b8OfRdSV6a5Argim7ZU7u9qO8n+UySh/dt//IkV3d7Wl9J8rhu+aOSbOr6+XaS13bLV3R93KV7fGCSc7tar0zyu31tn5zkrCTv6Nq/PMnUXP6Pktw9yXlJtib5Xnf/PgO2fX6STyd5U5Jt3Z7h46Ztdt9umxuSnN+F287n/2M3yrEtyaeSPHQuNc5VVf2wqk6uqi1VdUtVnQd8Hfj5bpNnApdX1T9W1Y+Ak4HDkxw6zDq0OBm0GkvdH+wnAVcOWH8M8EfAE4BDgMdP2+RvgR8C9wKO6247n/uzwEeBfwDuCRwLvDnJQ2Yp6XnA8cA+wNZdPH9g332eDjwaeEiSRwCnAy8Cfg54K3BuN5z5YOAE4JFVtQ/wRGBL18YbgDdU1d2ABwBnDaj93cA3gQOBZwN/keRX+9b/erfNvsC5wJtmeR363Ql4O3Bf4GBg+y6e+2jga8B+wEnA+5Lco2/9c4EX0HtN/xu9/9+dPkzv//mewBeAdw3qJMmbuw8sM90uncsPlmR/4EHA5d2ihwKX7FxfVT/sfpahBr4WJ4NW4+b9SW6gN4x3Hb0/yDP5DeDtVXVZ90fv5J0r0psE9CzgpKq6qaq+BJzZ99ynAluq6u1VdXNV/TtwNvCcWeo6o6our6qbgWMGPX8Ofe/0l1X13araTi/A31pVn6uqn1TVmcCPgV8AfgLsRS+Ql3R7XF/r2tgBPDDJflV1Y1X96/ROkhwE/DLw8qr6UVVdDPw98D/7Nruwqj7UHdN9J3D4LK/DrarqO1V1dvdz3gCsozf0Osh1wOurakdVvQf4CvCUvvVvr6qvdq/JWfSNZlTV6VV1Q1X9mJ/uTS4bUNf/qqp9B9wePtNz+iVZQi/Iz6yqL3eL9wa2Tdt0G70PXtKsDFqNm6d3e26PBQ6lt/czkwPphfFOV/XdX85Pj7nt1H//vsCj+/d0gN+itwc6yFyfv6u+B7X3smntHQQcWFVXAn9AL1yuS/LuJAd2z1tNb6/ry90Q9VNn6OdA4LtdEO50FXDvvsfX9t2/CfiZzOHYcZK7JnlrehOIfgB8Ctg3g2c7X123vYrJVV19g+rYu+vnzknWJ/la18+WbptB7407LL3j7+8E/oveSMJONwJ3m7b53YAbkHbBoNVYqqpPAmcAfz1gk2vohdFOB/fd3wrcDPQfL+zf9hvAJ6ft6exdVS+ZraQ5Pn9XfQ9qb9209u5aVRsAquofqupIeoFcwKu75VdU1Sp6w6mvBt7bDYv3+xZwjyT9e14HA1fP8rPO1cuABwOP7oavH9Mtz4Dt752kf93BXX278lzgafQODywDVszWT3pfA7txwO3ymZ7TPS/AacD+wLOqakff6svp29PvXucH8NOhZWkgg1bj7PXAE5LMNJR5FvD8JA9Jclf6hpi7IdD3ASd3e12Hctuh0vOAByV5XpIl3e2RSVbOsa6Bz59D3zN5G/DiJI9Oz88meUqSfZI8OMmvJtkL+BG946C3ACT57STLq+oW4PtdW7f0N1xV3wA+A/xlkp9Jb5LVauD/zeUHTW8C2hkDVu/T1fP97ljroGH+ne4J/F73ej0HWAl8aA5l7ENvKP07wF2Bv5ht4+5rYHsPuM12TPXvupp+rRu+7ncO8LAkz0ryM8CfAZf2DS1LAxm0GltVtRV4B70/atPXfZheEP8zvQlT/zxtkxPo7f1cS28ocAO9P9Z0w6hH05vE9K1um1fTOxY6l7p29fyBfQ9obxPwu/QmEn2v+3me363eC1gPXN+1d09++pWnY4DLk9xIb2LUsTMEBMAqenuB36IXGCdV1cfm8rPS2xv/9IB1rweWdrX9K/CRXbT1OXoTmq6ndzz32VX1nTnU8A56w8xXA1/q+hqq9L4T+yJ6x4Wv7dsD/i249b34rK7u79Gb2HVs3/PfkuQtw65Li0O88Lv2BEleDdyrqmaaAbxo+56PJP+N3kzbh08bRr0jbT0feGE3BC7tUdyj1aKU3vdcH94NxT6K3nDpOYu972Gqqv+qqpXzDVlpT+dZabRY7UNvyPZA4NvAa4AP7AF9SxozDh1LktSQQ8eSJDU0VkPH++23X61YsWLUZUiStFsuuuii66tq+UzrxipoV6xYwaZNm0ZdhiRJuyXJVYPWOXQsSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLU0Fhd+F2SNN5WrNm4W9tvWf+URpVMDoNWc+IvlyQY/Lu9Ys1Gf+8HMGg1JzP9AvmLJUm75jFaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqaF5B22Sg5JckORLSS5P8vvd8pOTXJ3k4u725PmXK0nSZBnGCStuBl5WVV9Isg9wUZKPduteV1V/PYQ+JEmaSPMO2qq6Brimu39Dks3AvefbriRJi8FQj9EmWQE8Avhct+iEJJcmOT3J3Qc85/gkm5Js2rp16zDLkSRp5IZ2ruMkewNnA39QVT9I8nfAq4Dq/n0N8DvTn1dVpwKnAkxNTdWw6tEdd/gp57Nt+445bTvXiw0sW7qES046ej5lSdJEGkrQJllCL2TfVVXvA6iqb/etfxtw3jD6Unvbtu8Y+sUCdvfqP5K0WAxj1nGA04DNVfXavuUH9G32DOCy+fYlSdKkGcYe7S8DzwO+mOTibtmfAKuSHEFv6HgL8KIh9CVJ0kQZxqzjC4HMsOpD821bkqRJ55mhJElqyKCVJKmhoX29R4vHPivXcNiZa4bcJsBwZzJL0iQwaHU7N2xe79d7JGlIHDqWJKkh92glSTPanbPEwdxGrvbEs8QZtJKkGXmWuOFw6FiSpIYMWkmSGjJoJUlqyGO0mtGwj6MsW7pkqO1J0qQwaHU7c538sGLNxqFPlJCkxcahY0mSGnKPVpI0I0/HOhwGrSRpRp6OdTgcOpYkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSHPDKU5GXQ2l0HLvdiAJPUYtJoTg1PaM3nJzPkzaCVJM9qdD9heNnMwj9FKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1NO+gTXJQkguSfCnJ5Ul+v1t+jyQfTXJF9+/d51+uJEmTZRh7tDcDL6uqhwC/ALw0yUOANcDHq+oQ4OPdY0mS9ijzDtqquqaqvtDdvwHYDNwbeBpwZrfZmcDT59uXJEmTZqjHaJOsAB4BfA7Yv6qu6VZdC+w/4DnHJ9mUZNPWrVuHWY4kSSM3tKBNsjdwNvAHVfWD/nVVVUDN9LyqOrWqpqpqavny5cMqR5KksTCUoE2yhF7Ivquq3tct/naSA7r1BwDXDaMvSZImyTBmHQc4DdhcVa/tW3UucFx3/zjgA/PtS5KkSTOM69H+MvA84ItJLu6W/QmwHjgryWrgKuA3htCXJEkTZd5BW1UXAhmw+nHzbV+SND5WrNm4W+u8GPxw9mglSXsIg3P3eQpGSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIUzBq0ZntXKwz8ZRykloyaLXoDArOFWs2GqqSFpxDx5IkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLU0F1GXYA0H4efcj7btu+Y8/Yr1mzc5TbLli7hkpOOnk9ZknQrg1YTbdv2HWxZ/5ShtjmXMJakuXLoWJKkhoYStElOT3Jdksv6lp2c5OokF3e3Jw+jL0mSJsmw9mjPAI6ZYfnrquqI7vahIfUlSdLEGErQVtWngO8Ooy1JkhaT1sdoT0hyaTe0fPeZNkhyfJJNSTZt3bq1cTmSJC2sVNVwGkpWAOdV1cO6x/sD1wMFvAo4oKp+Z7Y2pqamatOmTUOpR3uGw848rEm7Xzzui03albQ4JbmoqqZmWtfs6z1V9e2+At4GnNeqL+25bti83q/3SBprzYaOkxzQ9/AZwGWDtpUkabEayh5tkg3AY4H9knwTOAl4bJIj6A0dbwFeNIy+JEmaJEMJ2qpaNcPi04bRtiRJk8wzQ0mS1JBBK0lSQwatJEkNGbSSJDVk0EqS1JDXo9XEG/YJJpYtXTLU9iTt2QxaTbTdOSvUijUbh34WKUnaFYeOJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhL5OnRWe269POtM5L50lqyaDVomNwShonDh1LktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDQ0laJOcnuS6JJf1LbtHko8muaL79+7D6EuSpEkyrD3aM4Bjpi1bA3y8qg4BPt49liRpjzKUq/dU1aeSrJi2+GnAY7v7ZwKfAF4+jP4kaTGZ7dKOM/EKVZOl5WXy9q+qa7r71wL7z7RRkuOB4wEOPvjghuVI0niaKThXrNlooC4SCzIZqqoKqAHrTq2qqaqaWr58+UKUI0nSgmkZtN9OcgBA9+91DfuSJGkstQzac4HjuvvHAR9o2JckSWNpWF/v2QB8Fnhwkm8mWQ2sB56Q5Arg8d1jSZL2KMOadbxqwKrHDaN9SZImlWeGkiSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJauguoy5AkvYkh59yPtu275jTtivWbJzTdsuWLuGSk46eT1lqyKCVpAW0bfsOtqx/ylDbnGsgazQcOpYkqSGDVpKkhgxaSZIa8hitJGlR2t1j18M+dr6TQStJWpQGBeeKNRubhepMHDqWJKkhg1aSpIYMWkmSGjJoJUlqqPlkqCRbgBuAnwA3V9VU6z4laVzts3INh525ZshtAizc5B7tnoWadXxUVV2/QH1J0ti6YfN6T8G4h3HoWJKkhhYiaAs4P8lFSY6fvjLJ8Uk2Jdm0devWBShHkqSFsxBBe2RV/XfgScBLkzymf2VVnVpVU1U1tXz58gUoR5KkhdM8aKvq6u7f64BzgEe17lOSpHHRdDJUkp8F7lRVN3T3jwZe2bJPSdKe5/BTzmfb9h1z3n6uE8iWLV3CJScdfUfLAtrPOt4fOCfJzr7+oao+0rhPSdIeZtv2HU3OXzyMGd1Ng7aq/gM4vGUfkiSNM7/eI0lSQwatJEkNGbSSJDVk0EqS1NBCnetYkqRmWlysodcuzPeCDQatJGnitbhYAwzn6z0OHUuS1JBBK0lSQw4dS9ICG/b1Y5ctXTLU9jRcBq0kLaC5HkdcsWZjk2OOi9mwP8DAcD7EGLSSpIm3Ox9KFvpDjMdoJUlqyKCVJKkhg1aSpIYW3THa3TkY7kQDSVJriy5oZwpPZ+9JkkbFoWNJkhoyaCVJasiglSSpoUV3jFaSJJh9cuxM61rN5TFoJUmL0rhMgnXoWJKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqSGDVpKkhgxaSZIaMmglSWrIoJUkqaGJvajA4aecz7btO+a8/WxXcei3bOkSLjnp6DtalrSozfX3CMbnhO7SqDUP2iTHAG8A7gz8fVWtH0a727bvaPKLvDt/SKRh27BhA+vWrWPz5s2sXLmStWvXsmrVqlGXdauZfudWrNloqM7ToL87g5b7ek+WpkGb5M7A3wJPAL4JfD7JuVX1pZb9SpNow4YNrF27ltNOO40jjzySCy+8kNWrVwOMVdhq+AzOxS1V1a7x5BeBk6vqid3jVwBU1V/OtP397ne/Oumkk+bU9r/+x3f4hfv/3LBKbd6utCuf//znOeSQQ9h3331vXfb973+fK664gkc+8pGjK2wX/J2R4AUveMFFVTU107rWk6HuDXyj7/E3u2W3SnJ8kk1JNu3YMfdjrtJic9NNN7Fs2bLbLFu2bBk33XTTiCqSNAyt92ifDRxTVS/sHj8PeHRVnTDT9lNTU7Vp06Y5tX3YmYcNrc7pvnjcF5u1LQ3ysIc9jDe+8Y0cddRRty674IILOPHEE7nssstGWNnsPEYrQZKBe7StJ0NdDRzU9/g+3bJ5u2HzeidDaVFZu3Ytq1evvt0x2nXr1o26NEnz0DpoPw8ckuR+9AL2WOC5jfuUJtLOCU8nnnjirbOO161b50QoacI1DdqqujnJCcA/0ft6z+lVdXnLPqVJtmrVKoNVWmSaf4+2qj4EfKh1P5IkjSNPwShJUkMGrSRJDRm0kiQ1NLEXFZDUjhftkIbHoJV0O160QxqeiQ7aFr+0y5YuGXqbkqQ918QG7e582vYUcZKkUXEylCRJDRm0kiQ1NLFDx4MMOm4703KHkyVJrS26oDU8JUnjZNEFraT522flGg47c02DdgH8MKw9i0ErjZENGzawbt26Wy+Tt3bt2pFczcfrPUvDY9BKY2LDhg2sXbv2dhd+B7x0njTBnHUsjYl169Zx2mmncdRRR7FkyRKOOuooTjvtNNatWzfq0iTNg0ErjYnNmzdz5JFH3mbZkUceyebNm0dUkaRhMGilMbFy5UouvPDC2yy78MILWbly5YgqkjQMBq00JtauXcvq1au54IIL2LFjBxdccAGrV69m7dq1oy5N0jw4GUoaEzsnPJ144om3zjpet26dE6GkCWfQSmNk1apVBqu0yBi00gjt7vdKPfOZNHkMWmmEBgWnl3aUFg8nQ0mS1JBBK0lSQw4dS5pRi/MSL1u6ZOhtSuPOoJV0O7tzfNjjydLsHDqWJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIYMWkmSGjJoJUlqqFnQJjk5ydVJLu5uT27VlyRJ46r1maFeV1V/3bgPSZLGlkPHkiQ11DpoT0hyaZLTk9x9pg2SHJ9kU5JNW7dubVyOJEkLa15Bm+RjSS6b4fY04O+ABwBHANcAr5mpjao6taqmqmpq+fLl8ylHkqSxM69jtFX1+Llsl+RtwHnz6UuadIefcj7btu+Y8/ZzvUzdsqVLuOSko+9oWZIaazYZKskBVXVN9/AZwGWt+pImwbbtO5pcTq7FdWMlDU/LWcf/N8kRQAFbgBc17EuSpLHULGir6nmt2pakXdndPX0vXq9WWn+PVpJGYqbgXLFmo4GqBef3aCVJasiglSSpIYNWkqSGPEYrac4GTTCaabnHQqUeg1bSnBme0u5z6FiSpIbco5UWyD4r13DYmWsatAvgnqY0rgxaaYHcsHm9p2CU9kAOHUuS1JBBK0lSQw4dSwuoxTDvsqVLht6mpOExaKUFsjvHZz0nr7R4OHQsSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDzjqWNNEOP+V8tm3fMeft5/oVq2VLl3DJSUff0bKkWxm0kibatu07PLWlxppBO0K7+4vs9yolafIYtCM0U3B6ogJJWlwMWkkTzcsPatwZtJImmpcf1Ljz6z2SJDVk0EqS1JBBK0lSQwatJEkNGbSSJDVk0EqS1JBBK0lSQwatJEkNecKKBeDVRTTIbP/XM63z9JzS5DFoF4BXF9EgBqe0+M1r6DjJc5JcnuSWJFPT1r0iyZVJvpLkifMrU5KkyTTfPdrLgGcCb+1fmOQhwLHAQ4EDgY8leVBV/WSe/UnS7bQY3Vm2dMnQ29SeaV5BW1WbAZJMX/U04N1V9WPg60muBB4FfHY+/UnSdLsz/O5lKDUKrWYd3xv4Rt/jb3bLJEnao+xyjzbJx4B7zbBqbVV9YL4FJDkeOB7g4IMPnm9zkiSNlV0GbVU9/g60ezVwUN/j+3TLZmr/VOBUgKmpqboDfUmSNLZaDR2fCxybZK8k9wMOAf6tUV+SJI2t+X695xlJvgn8IrAxyT8BVNXlwFnAl4CPAC91xrEkaU8031nH5wDnDFi3Dlg3n/YXi31WruGwM9c0aBfAGZSSNM48M9QCuGHzes8MJUl7KC8qIElSQwatJEkNOXQsaVEadGhl0HLPGKVWDNoF4rlYpYVlcGpcGLQLwHOxStKey2O0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktTQXUZdwJ5sxZqNu7V8y/qntCxHktSAQTtCBqckLX4OHUuS1JBBK0lSQwatJEkNGbSSJDVk0EqS1JBBK0lSQwatJEkNGbSSJDVk0EqS1NC8gjbJc5JcnuSWJFN9y1ck2Z7k4u72lvmXKknS5JnvKRgvA54JvHWGdV+rqiPm2b4kSRNtXkFbVZsBkgynGkmSFpmWx2jvl+Tfk3wyyf8YtFGS45NsSrJp69atDcuRJGnh7XKPNsnHgHvNsGptVX1gwNOuAQ6uqu8k+Xng/UkeWlU/mL5hVZ0KnAowNTVVcy9dkqTxt8ugrarH726jVfVj4Mfd/YuSfA14ELBptyuUJGmCNbkebZLlwHer6idJ7g8cAvzHrp530UUXXZ/kqgYl7Qdc36DdFqy1nUmq11rbsNZ2JqneFrXed9CKeQVtkmcAbwSWAxuTXFxVTwQeA7wyyQ7gFuDFVfXdXbVXVcvnU88sdW6qqqldbzl61trOJNVrrW1YazuTVO9C1zrfWcfnAOfMsPxs4Oz5tC1J0mLgmaEkSWpoTwnaU0ddwG6w1nYmqV5rbcNa25mkehe01lT5jRpJklrZU/ZoJUkaCYNWkqSGFkXQJnl6kkpyaN+yRyX5RJIrknwhycYkh3XrTk5ydd/VhS5Osu8C1bq2u+LRpV2/j+7q/EpfLe/ttv2bJH827bl/uxB1dv39pKvnku41/KVu+Yru9f7zvm33S7IjyZu6xycn+aOFqnU36t55VakvJXlLkpH/Dgx4T5yQ5Mrudd5v1DXuNKDWd3Xv38uSnJ5kyajr3GlAvad1741Lk7w3yd6jrhNmrrVv3d8kuXGU9fUb8LqekeTrfX/Hjhh1nTCw1iRZl+SrSTYn+b2mRVTVxN+A9wD/ApzSPd4f2AL8Ut82RwJP7+6fDPzRCOr8ReCzwF7d4/2AA4FPAFMzbH83eif6uH93+zqw7wLWe2Pf/ScCn+zur+jq+ve+9S8BLgbeNMrXeA51X9bdvwvwKeCZo6hxDu+JR3T1bgH2G2WNc6j1yUC62wbgJaOudRf13q1vm9cCa8a11u7+FPDO/vf1ONYKnAE8e9T1zbHWFwDvAO7ULb9nyzqanBlqIXWfRo8EjgI+CJwEnACcWVWf2bldVV04mgpv4wDg+uqdopKquh4GX/2oqn6QZC3wpm7Rn1XV9xegzpncDfhe3+ObgM1JpqpqE/CbwFn03sTjZHrdAFTVzUk+Azxw4Uu6jRnfE8C3YOyujDVrrQBJ/g24zwhqm8mgegFI78VdCozDjNBBfxvuDPwV8FzgGaMr7zZ26+/YiA2q9SXAc6vqlm75dS2LGPmw2RA8DfhIVX0V2HkRg4cCX9jF8/6wb4jjguZV9pwPHNQNV7w5ya/0rXtXXz1/tXNhVW0A7k7vU/g7F6jOnZZ29XwZ+HvgVdPWvxs4NslBwE/o+4M7YruqmyR3BR4HfHGhi5tmtvfEuJm11m7I+HnAR0ZS3e0NrDfJ24FrgUPpnd1u1AbVegJwblVdM8LappvtfbCuG6J9XZK9RlVgn0G1PgD4zfSuHPfhJIe0LGIxBO0qen/w6f5dNX2DJJ/rxuHf0Lf4dVV1RHc7aiEKraobgZ8Hjge2Au9J8vxu9W/11fPHfbXfh96nsgNHcCxpe1fPocAxwDty24+tHwGeABxLb/h+XMxW9wOSXAx8GthYVR8eVZGwy/fEWJlDrW8GPlVV/zKC8m5ntnqr6gX0Rl820xuNGakBtf4J8BzG44PArWZ5XV9B74PLI4F7AC8fVY07zVLrXsCPqncaxrcBp7cuZGJv9P4zbwKuoncs6xvAf9Lbgzll2rbPBs7o7p/MiI4fzlDTBxlwjLbb5mzgOGA98FcLXN+N0x5/G7gntz3WeTq9PYN7AM9nzI7RDqp7XG873xN9j7cwJsdoZ6uV3iGb99Md8xrH2/TXtlv2GOC8Udc2oNYd3e/Wlu52C3DlqGub4+v62DF+XT8IfBm4X7cswLaW/U76Hu2zgXdW1X2rakVVHURvwtBHgeenm23auetIKuyT5MHThiiOoPchYdD2T6IXEO+g9+HhmUke0rTIwbUcCtwZ+M60Va8BXl5zuGjEKMxS91jY3ffEKA2qNckL6U06W1XdMa9xMKDe/0zywG59gF+n90d3pAbU+taqulf3t20FcFNVjXpOwWzvgwO69QGeDly28NXd1iy/X++nN68H4FeAr7asY9InQ60CXj1t2dnd8t8EXp3k3sB19C6J9Mq+7f4wyW/3PX56VW1pWCvA3sAb0/sq0c3AlfSGNN5L7xjt9m6764GnAq+nN4uvgB8m+WN6E6N+tXGdOy3thlmh96nvuOpd+vDWDarqcuDyBapnrnZZ9xiZ8T3Rfd3gfwP3Ai5N8qGqeuHoygQGv3+vpffH67Pda/y+qnrloEYW0Ez1vhg4J8nd6L03LqE3Y37UBr2242hQrWeld4nU0PsGwotHVWCfQbXeTO9v7h8CNwJNf7c8BaMkSQ1N+tCxJEljzaCVJKkhg1aSpIYMWkmSGjJoJUlqyKCVJKkhg1aSpIb+P8daNOAuwI7RAAAAAElFTkSuQmCC)
%% Cell type:markdown id: tags:
#### Lasso regularization
We can repeat the same process for Lasso regularization, which adds a penalty term which is proportional to the sum of the absolute values of the coefficients. Here the optimal value is alpha approximately 2.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# create two arrays for storage of the same size as alfas, but filled with zeros
lasso_training_score = np.zeros_like(alfas)
lasso_rsquared = np.zeros_like(alfas)
# loop over the values in the alfas array, at each loop the current value is alfa
# and idx is incremented by 1, starting at 0
for idx, alfa in enumerate(alfas):
lasso = Lasso(alpha=alfa) # instantatiate Lasso regularization with the current alfa
lasso.fit(X_train,Y_train) # train the model to our data set
# calculate the training score and store in the array lasso_training_score
lasso_training_score[idx] = lasso.score(X_train,Y_train)
preds_linear = lasso.predict(X_test) # calculate the model prediction for the test data
# calculate the correlation between the predicted and actual test data and store in the array lasso_rsquared
lasso_rsquared[idx] = r2_score(Y_test,preds_linear)
plt.scatter(alfas,lasso_training_score,label='Training score') # plot the training score against alpha
plt.scatter(alfas,lasso_rsquared,label='Testing score') # plot the testing score against alpha
plt.xscale('log') # make the x-axis a logarithmic scale
plt.gca().set_xlim(left=.01, right=100); # fix the x-axis limits
plt.title('Lasso regularization')
plt.xlabel('alpha');
plt.legend(loc='best');
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAX0AAAEaCAYAAAD9iIezAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkvUlEQVR4nO3dfXxU5Z338c+P8BSEEgu4XRItuEUsYAglguhaVLSgVlFWKUoV7+oiWk3XrlqpraVUVyx7+4BVWVrRVm0VLSJWurhKvX3AiqEogsCCSEviQ5GWCBIk4O/+Y07CMCRkJnMmM5nzfb9eeWXmPFxzzZXke65c58x1zN0REZFoaJftCoiISOtR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EWSZGbTzOzhNPafbWY/DLNOQbm/N7NJYZcr+al9tisgbYOZbQIuc/fnsl2Xtsrdp6RbhplNA77k7t+MK/f0dMuV6FBPX/KGmeVsJ8bMCrJdBxFQ6EuazOxQM/udmW0xs78Hj0vi1l9iZhvNbLuZvWtmE4PlXzKz/2dmNWb2kZk9FrfP8Wb2erDudTM7/iCvv8nMvmdmK4FPzKy9mR1nZkvNbJuZvWlmJ8Vt39fMXgzq85yZ3VM/ZGNmJ5lZVSPln9rEaz9uZh8E9XzRzAbGrXvQzO4zs0Vm9glwcrDs5mD902a2I+7rMzO7JFh3l5ltNrOPzWy5mZ0YLB8DfB/4RrDPm8HyF8zssuBxOzP7gZn92cz+ama/MrPuwbo+ZuZmNsnM/hK0+43N/pAlryj0JV3tgAeALwJHALXAzwDM7BBgFnC6u3cDjgfeCPb7CfAscChQAtwd7PN54Jlgvx7A7cAzZtbjIHW4ADgTKAL+Idj/ZuDzwLXAb82sV7Dtr4FlQdnTgIta/tb5PdAPOAz4E/BIwvoLgVuAbsDL8Svc/Sx37+ruXYHzgQ+A54PVrwNlQf1/DTxuZp3d/b+B/wAeC/Yd3EidLgm+TgaOBLoS/Dzi/DPQHxgF3GRmX07pXUubptCXtLj7Vnf/rbvvdPftxEJuZNwmnwGDzKzQ3d9399XB8jpiB4re7r7L3etD8Uxgvbs/5O573P03wFrgrINUY5a7b3b3WuCbwCJ3X+Tun7n7/wCVwBlmdgRwLHCTu+8OXnNhGu99rrtvd/dPiR1ABtf3qgNPufsrQT12NVaGmR0F/BIY7+6bg3IfDtp1j7v/X6ATsZBOxkTgdnff6O47gKnAhIShrx+7e627vwm8CTR28JA8pdCXtJhZFzP7r2A44WPgRaDIzArc/RPgG8AU4H0ze8bMjg52vR4wYJmZrTazbwXLewN/TniZPwPFB6nG5rjHXwTOD4Z2tpnZNmI9238Myv6bu+9sYt+kmVmBmc0ws3eC970pWNUz2bKDA8RTwA/iDnqY2bVmtiYYNtoGdE8o92AS2+/PxC7Y+Ie4ZR/EPd5J7L8BiQiFvqTr34n1Qoe7++eArwbLDcDdF7v7acRCdy3w82D5B+7+r+7eG7gcuNfMvgS8Ryy44x0BVB+kDvFTxW4GHnL3orivQ9x9BvA+8Hkz6xK3/eFxjz8BGtYFJ1970bgLgbHAqcRCuU/8+26kXvsxs3bEhm7+4O5z4pafSOyAOB441N2LgJq4cpubFjex/Y4A9gAfNrOfRIRCX1LRwcw6x321JzZeXQtsC8bjf1S/sZn9g5mNDcb2PwV2EBvuwczOt30nfP9OLMw+AxYBR5nZhcFJ2W8AA4DfJVnHh4GzzGx00BvvHJygLXH3PxMb6plmZh3NbAT7Dxv9L9DZzM40sw7AD4gNrTSmW/CethI7UPxHkvWrdwtwCPCdRsrdA2wB2pvZTcDn4tZ/CPQJDhqN+Q1wTXDCuiv7zgHsSbF+kqcU+pKKRcQCvv5rGnAnUAh8BPwR+O+47dsB3yXW+/wbsbH+K4J1xwKvmdkOYuPq3wnGobcCXyf2H8RWYr3er7v7R8lUMBgXH0vsKpctxHr+17Hvd30iMCIo+2bgMWLhjbvXAFcCvyD2n8UnwH5X88T5FbGhk2rg7eC9p+IC4Djg73FX8EwEFhNrw/8Nyt/F/sNEjwfft5rZnxopdy7wELFhtneD/a9OsW6Sx0w3UZEos9ilomvd/UfNbiySB9TTl0gxs2PN7J+C69nHEPuvYEGWqyXSanL2E4wiGfIFYD6x6/SrgCvcfUV2qyTSejS8IyISIRreERGJEIW+iEiEZG1Mv2fPnt6nT59svbyISJu0fPnyj9y9qQ8NNitrod+nTx8qKyuz9fIiIm2SmSVOU5KSpIZ3zGyMma0zsw1mdkMj6y+x2NS6bwRfl6VTKRERyYxme/rB/CP3AKcRu8TtdTNb6O5vJ2z6mLtflYE6iohISJLp6Q8DNgQfkd8NPErsAy0iItLGJBP6xew/90cVjU9z+y9mttLMnjCzwxtZLyIiWRbWidyngd+4+6dmdjmxm0KckriRmU0GJgMUfK4XJ8xYwnWj+3POkINNlX6gBSuqmbl4He9tq6V3UWFWygijDiIirS2Z0K9m/znHS0iY2zyYGbHeL4CfNlZQMG/4HIBO/9jPq7fVMnX+WwBJB+aCFdVMnf8WtXV7Y5XLQhlh1KG+nGwfvMIoQwdAkbYjmeGd14F+wfzcHYEJJNxizsz+Me7p2cCaZCtQW7eXmYvXJbs5MxevawjbbJURRh3qDxzV22px9h04Fqw42L1Ccq+MMOpQX84JM5bQ94ZnOGHGkpT3D6sMkXzXbOgHN1+4itg832uAee6+2symm9nZwWYVwS3v3gQqiN2YOWnvbatNe9vWLCOMOuTCwSuMMvLtAKiDhuS7pMb03X0RsRtoxC+7Ke7xVGI3YG6R3kWFKW1b3Ui4tmYZYdQhFw5eYZSR6QNgssNE6ZaRS0N2IpmU9bl3CjsUcN3o/klvf93o/hR2KMhqGWHUoakDRKoHr2yXEUYdcuHglSv/sYhkWlZDv7iokFvHHZNST+icIcXcOu4YiosKsSyVEUYdcuHgFUYZ+XIAzJUhO5FMy9rcO8cUd+eVGw64qjMp5wwpTvtf5nTLCGN/IK2hgFwoI4w6XDe6/35DK9Cyg1c6ZeTKkJ1IpmXtJirl5eWuCdekXrYvPU0c04fYQSOV/+BOmLGk0QNHcVFhSh0cnReQgzGz5e5e3uL9FfoiMWF8XiHdA0cYZUh+Szf0dY9ckUAuDNmFcSWTyMEo9EVClO6BQ+cFJNMU+iI5JIwTyqDzAtK0rF+nLyL7hHEJrD4vIAej0BfJIWF8BkSfF5CD0fCOSI7ReQHJJPX0RfJMGJ9wlvyl0BfJM2GcF5D8peEdkTwTxucFJH8p9EXyUBjzU0l+UuiLyAF0nX/+UuiLyH7CuqGM5CadyBWR/eg6//ym0BeR/eg6//ym0BeR/eg6//ym0BeR/eg6//ymE7kiuWblPHh+OtRUQfcSGHUTlI5vtZfXdf75TaEvkktWzoOnK6AuGD+v2Rx7Dq0e/Ar5/KThHckfK+fBHYNgWlHs+8p5rbt/GGU8P31f4Nerq40tFwmBevqSG9Id0ki3hxxGDzuMMmqqUlsukiL19CX76sOyZjPg+8IylV5yuj3kMHrYYZTRvSS15U0J478WyUsKfUlfLgxppNtDDqOHHUYZo26CDgmXRnYojC1PVhgHUclbCn1JTxgBE0ZYpttDDqOHHUYZpePhrFnQ/XDAYt/PmpXaUJfOC8hBKPQlPbkypJFuDzmMHnYYZUAs4K9ZBdO2xb6netWOzgvIQSj0Jb3hmVwZ0ki3hxxGDzuMMsIQ1nkByUvm7ll54fLycq+srMzKa4cijA/Q5EIZiVecQCxwkw2rOwYFQzsJuh8e66WmUo8sfiApr6T7Mw2JpmfODDNb7u7lLd4/mdA3szHAXUAB8At3n9HEdv8CPAEc6+4HTfTy3u298rqj22ZYhvFHlStlpBvaORIwkiCMzkAa+ydOzwyxqRxuHXeMgj9NGQ99MysA/hc4DagCXgcucPe3E7brBjwDdASuaj70C7xycte2GZZh9G5zpYxpRUBjvwMWG1NOhnrp+SWEv7ETZiyhupFZOYuLCnnlhlPCqmkkpRv6yYzpDwM2uPtGd98NPAqMbWS7nwC3AbtSqkE2roVOt4xcubwvF656gfRPPEpuCeFvTNMz565kQr8YiO9OVgXLGpjZV4DD3f2ZFtWirYVlrlzelwtXvUj+CeFvTNMz5660r94xs3bA7cC/J7HtZDOrNLPKLTvjhhTaWljmyuV9uXDVi+SfEP7GND1z7kom9KuBw+OelwTL6nUDBgEvmNkm4DhgoZkdMObk7nPcvdzdy3t1sdjCthiWuXJ5X1iBreEZiRfC39g5Q4q5ddwxFBcVYsTG8nUSNzckcyK3PbETuaOIhf3rwIXuvrqJ7V8Ars3rq3dE8p3+PnJWa12yeQZwJ7FLNue6+y1mNh2odPeFCdu+QDKh39av0xcRyYJ0Qz+pqZXdfRGwKGFZo//ruftJLa2MiOQR/beQkzSfvoiEL0fuACYH0tw7IhI+zfSZsxT6IhI+zfSZsxT6IhI+zfSZsxT6IhI+fdI7Zyn0RSR8+qR3ztLVOyKSGaXj0w55zckfPoW+iOSkxDn5q7fVMnX+WwAK/jRoeEdEctLMxev2uwkLQG3dXmYuXpelGuUHhb6I5CTNyZ8ZCn0RyUmakz8zFPoikpM0J39m6ESuiOSk+pO1unonXAp9EclZ5wwpVsiHTMM7IiIRotAXEYkQhb6ISIQo9EVEIkShLyK5a+U8uGMQTCuKfV85L9s1avN09Y6I5CbdcjEj1NMXkdykWy5mhEJfRHKTbrmYEQp9EclNuuViRij0RSQ36ZaLGaHQF5HcpFsuZoSu3hGR3BXCLRdlf+rpi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCQV+mY2xszWmdkGM7uhkfVTzOwtM3vDzF42swHhV1VERNLVbOibWQFwD3A6MAC4oJFQ/7W7H+PuZcBPgdvDrqiIiKQvmZ7+MGCDu290993Ao8DY+A3c/eO4p4cAHl4VRUQkLMl8OKsY2Bz3vAoYnriRmX0b+C7QETgllNqJiEioQjuR6+73uPs/Ad8DftDYNmY22cwqzaxyy5YtYb20iIgkKZnQrwYOj3teEixryqPAOY2tcPc57l7u7uW9evVKupIiIhKOZEL/daCfmfU1s47ABGBh/AZm1i/u6ZnA+vCqKCIiYWl2TN/d95jZVcBioACY6+6rzWw6UOnuC4GrzOxUoA74OzApk5UWEZGWSWqWTXdfBCxKWHZT3OPvhFwvERHJAH0iV0QkQhT6IiIRotAXEYkQ3TlLRPLaghXVzFy8jve21dK7qJDrRvfnnCHF2a5W1ij0RSRvLVhRzdT5b1FbtxeA6m21TJ3/FkBkg1/DOyKSt2YuXtcQ+PVq6/Yyc/G6LNUo+xT6IpK33ttWm9LyKFDoi0je6l1UmNLyKFDoi0jeum50fwo7FOy3rLBDAdeN7p+lGmWfTuSKSN6qP1mrq3f2UeiLSF47Z0hxpEM+kYZ3REQiRKEvIhIhCn0RyW8r58Edg2BaUez7ynnZrlFWaUxfRPLXynnwdAXUBdfl12yOPQcoHZ+9emWRevoikr+en74v8OvV1caWR5RCX0TyV01VassjQKEvIvmre0lqyyNAoS8i+WvUTdAhYcqFDoWx5RGl0BeR/FU6Hs6aBd0PByz2/axZkT2JC7p6R0TyXen4SId8IvX0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIUmFvpmNMbN1ZrbBzG5oZP13zextM1tpZs+b2RfDr6qIiKSr2dA3swLgHuB0YABwgZkNSNhsBVDu7qXAE8BPw66oiIikL5me/jBgg7tvdPfdwKPA2PgN3P0P7r4zePpHILqTVYuI5LBkQr8Y2Bz3vCpY1pRLgd+nUykREcmMUKdWNrNvAuXAyCbWTwYmAxxxxBFhvrSIiCQhmZ5+NXB43POSYNl+zOxU4EbgbHf/tLGC3H2Ou5e7e3mvXr1aUl8REUlDMqH/OtDPzPqaWUdgArAwfgMzGwL8F7HA/2v41RQRkTA0G/ruvge4ClgMrAHmuftqM5tuZmcHm80EugKPm9kbZrawieJERCSLkhrTd/dFwKKEZTfFPT415HqJiEgG6BO5IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEIU+iIiEaLQFxGJEIW+iEiEKPRFRCJEoS8iEiEKfRGRCFHoi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiERIUqFvZmPMbJ2ZbTCzGxpZ/1Uz+5OZ7TGz88KvpoiIhKHZ0DezAuAe4HRgAHCBmQ1I2OwvwCXAr8OuoIiIhKd9EtsMAza4+0YAM3sUGAu8Xb+Bu28K1n2WgTqKiGTNghXVzFy8jve21dK7qJDrRvfnnCHF2a5WiyUT+sXA5rjnVcDwzFRHRCR3LFhRzdT5b1FbtxeA6m21TJ3/FkCbDf5WPZFrZpPNrNLMKrds2dKaLy0ikrKZi9c1BH692rq9zFy8Lks1Sl8yoV8NHB73vCRYljJ3n+Pu5e5e3qtXr5YUISLSat7bVpvS8rYgmeGd14F+ZtaXWNhPAC7MaK1ERHJA76JChn78P1zffh697SPe8578dM94ln/utGxXrcWa7em7+x7gKmAxsAaY5+6rzWy6mZ0NYGbHmlkVcD7wX2a2OpOVFhFpDXcOWM9tHX5BSbuPaGdQ0u4jbuvwC+4csD7bVWsxc/esvHB5eblXVlZm5bVFRJJyxyCo2Xzg8u6HwzWrWr8+gJktd/fylu6vT+SKiDSlpiq15W2AQl9EpCndS1Jb3gYo9EVEmjLqJuhQuP+yDoWx5W2UQl9EpCml4+GsWbExfCz2/axZseVtVDKXbIqIRFfp+DYd8onU0xcRiRCFvohIhCj0RUQiJKfG9Ovq6qiqqmLXrl3Zroo0oXPnzpSUlNChQ4dsV0VEWiCnQr+qqopu3brRp08fzCzb1ZEE7s7WrVupqqqib9++2a6OiLRATg3v7Nq1ix49eijwc5SZ0aNHD/0nJtKG5VToAwr8HKefj0jblnOhLyIimaPQj7N161bKysooKyvjC1/4AsXFxQ3Pd+/efdB9KysrqaioaPY1jj/++LCqKyKSspw6kZuqsG9Y3KNHD9544w0Apk2bRteuXbn22msb1u/Zs4f27RtvsvLycsrLm5/tdOnSpS2uXyYd7L2JSP5osz39+hsWV2+rxdl3w+IFK1p0J8cmXXLJJUyZMoXhw4dz/fXXs2zZMkaMGMGQIUM4/vjjWbcudq/MF154ga9//etA7IDxrW99i5NOOokjjzySWbNmNZTXtWvXhu1POukkzjvvPI4++mgmTpxI/b0NFi1axNFHH83QoUOpqKhoKDfe6tWrGTZsGGVlZZSWlrJ+feymDr/61a8oLS1l8ODBXHTRRQBs2rSJU045hdLSUkaNGsVf/vKXRt/bO++8w5gxYxg6dCgnnngia9euDbUtRST72mzX7mA3LA77LvVVVVUsXbqUgoICPv74Y1566SXat2/Pc889x/e//31++9vfHrDP2rVr+cMf/sD27dvp378/V1xxxQHXtq9YsYLVq1fTu3dvTjjhBF555RXKy8u5/PLLefHFF+nbty8XXHBBo3WaPXs23/nOd5g4cSK7d+9m7969rF69mptvvpmlS5fSs2dP/va3vwFw9dVXM2nSJCZNmsTcuXOpqKhgwYIFB7y3UaNGMXv2bPr168drr73GlVdeyZIlS0JtSxHJrjYb+q15w+Lzzz+fgoICAGpqapg0aRLr16/HzKirq2t0nzPPPJNOnTrRqVMnDjvsMD788ENKSvafg3vYsGENy8rKyti0aRNdu3blyCOPbLgO/oILLmDOnDkHlD9ixAhuueUWqqqqGDduHP369WPJkiWcf/759OzZE4DPf/7zALz66qvMnz8fgIsuuojrr7/+gPe2Y8cOli5dyvnnn9+w7tNPP21Re4lI7mqzwzu9iwpTWp6OQw45pOHxD3/4Q04++WRWrVrF008/3eQ16506dWp4XFBQwJ49e1q0TVMuvPBCFi5cSGFhIWeccUaLe+T17+2zzz6jqKiIN954o+FrzZo1LSpTRHJXmw3960b3p7BDwX7LCjsUcN3o/hl93ZqaGoqLY8NHDz74YOjl9+/fn40bN7Jp0yYAHnvssUa327hxI0ceeSQVFRWMHTuWlStXcsopp/D444+zdetWgIbhneOPP55HH30UgEceeYQTTzzxgPI+97nP0bdvXx5//HEg9unbN998M+y3JyJZ1mZD/5whxdw67hiKiwoxoLiokFvHHRP6eH6i66+/nqlTpzJkyJCUeubJKiws5N577204odqtWze6d+9+wHbz5s1j0KBBlJWVsWrVKi6++GIGDhzIjTfeyMiRIxk8eDDf/e53Abj77rt54IEHKC0t5aGHHuKuu+5q9LUfeeQR7r//fgYPHszAgQN56qmnQn9/IpJdVn/FSGsrLy/3ysrK/ZatWbOGL3/5y1mpTy7ZsWMHXbt2xd359re/Tb9+/bjmmmuyXa0G+jmJZI+ZLXf35q8Pb0Kb7enns5///OeUlZUxcOBAampquPzyy7NdJRHJE2326p18ds011+RUz15E8od6+iIiEaLQFxGJEIW+iEiEKPRFRCJEJ3LjbN26lVGjRgHwwQcfUFBQQK9evQBYtmwZHTt2POj+L7zwAh07dmyYPnn27Nl06dKFiy++OLMVFxFJUtsO/ZXz4PnpUFMF3Utg1E1QOr7FxTU3tXJzXnjhBbp27doQ+lOmTGlxXTLJ3XF32rXTP3oiUdN2/+pXzoOnK6BmM+Cx709XxJaHaPny5YwcOZKhQ4cyevRo3n//fQBmzZrFgAEDKC0tZcKECWzatInZs2dzxx13UFZWxksvvcS0adP4z//8TwBOOukkvve97zFs2DCOOuooXnrpJQB27tzJ+PHjGTBgAOeeey7Dhw8n8UNrADfccEPD69UfiD788EPOPfdcBg8ezODBgxvm6r/99tsZNGgQgwYN4s477wRi0yv379+fiy++mEGDBrF582ZmzpzJscceS2lpKT/60Y9CbTcRyU1tt6f//HSoS5hRs642tjyN3n48d+fqq6/mqaeeolevXjz22GPceOONzJ07lxkzZvDuu+/SqVMntm3bRlFREVOmTNnvv4Pnn39+v/L27NnDsmXLWLRoET/+8Y957rnnuPfeezn00EN5++23WbVqFWVlZQfUY+vWrTz55JOsXbsWM2Pbtm0AVFRUMHLkSJ588kn27t3Ljh07WL58OQ888ACvvfYa7s7w4cMZOXIkhx56KOvXr+eXv/wlxx13HM8++yzr169n2bJluDtnn302L774Il/96ldDaTsRyU1J9fTNbIyZrTOzDWZ2QyPrO5nZY8H618ysT+g1TVRTldryFvj0009ZtWoVp512GmVlZdx8881UVcXKLy0tZeLEiTz88MNJ33Fq3LhxAAwdOrRhQrWXX36ZCRMmADBo0CBKS0sP2K979+507tyZSy+9lPnz59OlSxcAlixZwhVXXAHEZuns3r07L7/8Mueeey6HHHIIXbt2Zdy4cQ3/VXzxi1/kuOOOA+DZZ5/l2WefZciQIXzlK19h7dq1DTdiEZFwLVhRzQkzltD3hmc4YcaSFt3sqb6Mjl/40tB06tJsWplZAXAPcBpQBbxuZgvd/e24zS4F/u7uXzKzCcBtwDfSqVizupcEQzuNLA+JuzNw4EBeffXVA9Y988wzvPjiizz99NPccsstvPXWW82WVz+VcqrTKLdv355ly5bx/PPP88QTT/Czn/2sRVMpx08R7e5MnTpVUzyIZNiCFdW8/OS9PMaj9O70Ee/t7MmdT04Arkx6gsj4Ms6xTWnVJ5me/jBgg7tvdPfdwKPA2IRtxgK/DB4/AYwyM0urZs0ZdRN0SJg7v0NhbHlIOnXqxJYtWxpCv66ujtWrV/PZZ5+xefNmTj75ZG677TZqamrYsWMH3bp1Y/v27Sm9xgknnMC8ebHzEG+//XajB48dO3ZQU1PDGWecwR133NEw5fGoUaO47777ANi7dy81NTWceOKJLFiwgJ07d/LJJ5/w5JNPNjqV8ujRo5k7dy47duwAoLq6mr/+9a8p1V1EmvfGM3OYbnMoafcR7QxK2n3EdJvDG88ceHOkZMpIVzLjEsVAfJe6Chje1DbuvsfMaoAewH41NLPJwGSAI444ooVVDtSP24d49U6idu3a8cQTT1BRUUFNTQ179uzh3/7t3zjqqKP45je/SU1NDe5ORUUFRUVFnHXWWZx33nk89dRT3H333Um9xpVXXsmkSZMYMGAARx99NAMHDjxgKuXt27czduxYdu3ahbtz++23A3DXXXcxefJk7r//fgoKCrjvvvsYMWIEl1xyCcOGDQPgsssuY8iQIQ3DSfW+9rWvsWbNGkaMGAHE7t378MMPc9hhh6XZaiIS77LdD9Ol3e79lnWx3Vy2+2Hgxy0uo6WanVrZzM4Dxrj7ZcHzi4Dh7n5V3Dargm2qgufvBNs0eVjS1Moxe/fupa6ujs6dO/POO+9w6qmnsm7dumY/E5BNUfw5ibTUZ9OKaMeBOfsZRrtp21Iuo3zODirf29vikZRkevrVwOFxz0uCZY1tU2Vm7YHuwNaWVipKdu7cycknn0xdXR3uzr333pvTgS8iqdlV+AW61L7f+PI0y2iJZEL/daCfmfUlFu4TgAsTtlkITAJeBc4Dlni27s7SxnTr1q3R6/JFJD90OX06e566mvZ7991Pe09BZ7qcPj2tMlqq2dAPxuivAhYDBcBcd19tZtOBSndfCNwPPGRmG4C/ETswtIi7k+lzwNJyOpaLpKh0fCxo484/tk/1/GN8GaxJqzo5dbvEd999l27dutGjRw8Ffw5yd7Zu3cr27dvp27dvtqsjEknp3i4xpz6RW1JSQlVVFVu2bMl2VaQJnTt3pqQkvM9CiEjryqnQ79Chg3qQIiIZ1HYnXBMRkZQp9EVEIkShLyISIVm7esfMtgPrMvwy3YGaDO+bzHZNbZPK8sRlic97kjDtRQbkQnsebH2y7dlc+7ZGWzZVj7D3C7s9o/y7mcy2rfG33t/duzVf1SbU30Wptb+IXeOf6deYk+l9k9muqW1SWZ64rJHnkWjPg61Ptj2ba9/WaMt02jOV/cJuzyj/bqbTnrn0t57vwztPt8K+yWzX1DapLE9cls57a6lcaM+DrU+2PZNp39bQ0tdMZb+w2zPKv5vJbJvzf+vZHN6p9DQ+YCD7U3uGR20ZLrVnuNJtz2z29JOfTFqSofYMj9oyXGrPcKXVnlnr6YuISOvL9zF9ERGJo9AXEYkQhb6ISITkZOib2Tlm9nMze8zMvpbt+rR1Znakmd1vZk9kuy5tkZkdYma/DH4nJ2a7Pm2dfh/DlWpehh76ZjbXzP4a3Dc3fvkYM1tnZhvM7IaDleHuC9z9X4EpwDfCrmNbElJ7bnT3SzNb07YlxXYdBzwR/E6e3eqVbQNSaU/9PjYvxfZMKS8z0dN/EBiTUNEC4B7gdGAAcIGZDTCzY8zsdwlfh8Xt+oNgvyh7kPDaU/Z5kCTbldh9oTcHm+1txTq2JQ+SfHtK8x4k9fZMKi9Dn0/f3V80sz4Ji4cBG9x9I4CZPQqMdfdbga8nlmGx22bNAH7v7n8Ku45tSRjtKQdKpV2BKmLB/wY5OiSabSm259utXL02J5X2NLM1pJCXrfULXMy+nhLE/oiKD7L91cCpwHlmNiWTFWujUmpPM+thZrOBIWY2NdOVa8Oaatf5wL+Y2X1kZ4qBtqrR9tTvY4s19fuZUl7m1J2z6rn7LGBWtuuRL9x9K7HxPmkBd/8E+D/Zrke+0O9juFLNy9bq6VcDh8c9LwmWScuoPTND7RoutWe4QmnP1gr914F+ZtbXzDoCE4CFrfTa+UjtmRlq13CpPcMVSntm4pLN3wCvAv3NrMrMLnX3PcBVwGJgDTDP3VeH/dr5SO2ZGWrXcKk9w5XJ9tSEayIiEaLLz0REIkShLyISIQp9EZEIUeiLiESIQl9EJEIU+iIiEaLQl0gys01m1jPdbUTaGoW+iEiEKPQl75nZAjNbbmarzWxywro+ZrbWzB4xszVm9oSZdYnb5Goz+5OZvWVmRwf7DDOzV81shZktNbP+rfqGRNKg0Jco+Ja7DwXKgQoz65Gwvj9wr7t/GfgYuDJu3Ufu/hXgPuDaYNla4ER3HwLcBPxHRmsvEiKFvkRBhZm9CfyR2CyF/RLWb3b3V4LHDwP/HLdufvB9OdAneNwdeDy4ld0dwMBMVFokExT6ktfM7CRiN5gY4e6DgRVA54TNEiegin/+afB9L/vuP/ET4A/uPgg4q5HyRHKWQl/yXXfg7+6+MxiTP66RbY4wsxHB4wuBl5Mos34e80tCqaVIK1HoS777b6B93H1E/9jINuuAbwfbHEps/P5gfgrcamYryNG7z4k0RVMrS6QFN5/+XTBUI5L31NMXEYkQ9fRFRCJEPX0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIT8f35uD39vW5dJAAAAAElFTkSuQmCC)
%% Cell type:markdown id: tags:
Again, it can be seen that the most significant variables are `BMI`, `BP` and `S5`. In this case the coefficients for `AGE`, `S2` and `S4` are zero or close to zero.
%% Cell type:code id: tags:
``` python
rng = np.random.RandomState(1) # make sure the results are repeatable
# cross_validate takes the particular model, in this case lasso regularization
# and undertakes a number of runs according the method specified by cv=
# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times, giving 25 runs
# return_estimator=True returns the fitting data for each run
scores = cross_validate(Lasso(alpha=2.), nX, Y, cv=RepeatedKFold(n_splits=5, n_repeats=5), return_estimator=True)
# take the results for each simulation (estimator), extract the coefficients for each run
# and add them to a dataframe with columns being the feature names
coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)
# plot the descriptive statics of the coefficients in a box and whisker plot to show variability
ax = coefs.plot(kind='box',figsize=(10,7))
plt.title('Lasso regression, alpha = 2.')
plt.axhline(y=0, color='.5')
plt.subplots_adjust(left=.3)
```
%%%% Output: display_data
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAAGrCAYAAABnmlbLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAglklEQVR4nO3de3hldX3v8ffXIcyMCoMIKleD4iUaHmhNvZ1RmVbx1hZq9eh4OWhT6fiUaU9FSyTnOUBPY8d6qrZRnynteBxvUY5VtI4XbBtLU28MCgrECwcHHYVyEUaQAQJ8zx9rzXRPyM4kZP+y907er+fZT/Zea2Wtb/be2Z+9fuu3fisyE0mSVMZD2l2AJElLmUErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq20xEXEcyLi+x1Qx46IeH6rl5U6nUGrjuWHbWtk5r9l5pPaXUc3iYhHRcRYRPwsInZFxL9HxDPaXZe6k0ErLUBEHNDJ69OD9nDgUuBpwKHAVmBbRDy8rVWpKxm06joR8YiI+FxE3BQRt9b3j26Y//qIuDYibo+IH0XEa+rpx0fEv9Z7KDdHxCcafufZEXFpPe/SiHj2LNvfERFnR8R3gF9GxAER8cyI+GpE3BYRV0TEyQ3LHxcRl9T1/FNEvC8iPlLP642IjIjBiPgx8C/19N+LiMn67/tSRDy2nh4R8e6IuDEifhER342I/nreSyLi6no7P42It9TTT46InQ319EXEV+par4qI326Y98G6vm31er4REY+f4+vy+Ij4l4i4pX5+PxoRhzRZ9ryI+GREfKLezrci4sRpi50UEd+pX5NPRMSq+ndnff1bITOvzcx3Zeb1mXlfZl4AHAjYMqD5y0xv3jryBuwAnj/D9EcCvws8FDgI+L/ARfW8hwG/AJ5UPz4CeGp9fwwYpvqCuQpYW08/FLgVeB1wALC+fvzIWeq6HDgGWA0cBdwCvKRe9wvqx4fXy38N+N9UH9Rr6/o+Us/rBRL4UF37auBU4Bqgr67nfwBfrZd/IXAZcAgQ9TJH1POuB55T338E8Kv1/ZOBnfX9nnrd59T1/Dpwe8Pz9cG69qfX2/4o8PE5vl7H13/7SuBw4BLgPTO9nsB5wBTw8rqmtwA/Anoalv0mcGT9+kwCG/b3+jep63PAbU1un5vj33YScBewpt3/F96679b2Arx5a3ajSdDOsNxJwK31/YfVH6C/C6yettyHgAuAo6dNfx3wzWnTvga8fpa6fq/h8dnAh6ct8yXgdOBY4F7goQ3zPsIDg/ZxDfO/AAw2PH4IcCfw2DoYfwA8E3jItG3+GPgD4OBp00/mP4P2OcANjb9L9QXkvPr+B4G/b5j3EuB7D/L1Ow349kyvJ1XQfn3a39j4RWEH8NqG+X8JbN7f61/ofXgw8F3gbe36X/DW3TebjtV1IuKhEfG3EXFdRPyCas/pkIhYkZm/BF4JbACur5tAn1z/6p9S7QV+s24y/b16+pHAddM2cx3VnmozP2m4/1jgFXVT7G0RcRvVnusR9bp/npl3NvndZuv764Z1/byu+6jM/BfgvcD7gBsj4oKIOLj+vd+lCsbr6ibyZ82wnSOBn2Tm/bP8rTc03L+T6njlfkXEoyPi43Wz9S+ovlAcNsuv7P2b63p21vXNWsdsr/9c6pyPiFgN/CPVl4K/aPX6tTwYtOpGZ1EdK3tGZh4MPLeeHgCZ+aXMfAFV0H0P+Lt6+g2Z+cbMPJJqz+/9EXE88DOqcGt0LPDTWWpovOzVT6j2aA9puD0sMzdR7aUdGhEPbVj+mDms7w+mrW91Zn61/jv+JjOfBjwFeCLw1nr6pZl5KvAo4CLgwhm28zPgmIho/N/f3986V2+v/44T6tfltdSvSRN7n4e6nqPr+vZn1td/uoj4QkTc0eT2hWYbiYiVVM/jTqr3i/SgGLTqdD0RsarhdgDVcbndwG0RcShw7p6F672qUyPiYcDdwB3A/fW8VzR0mrmVKhTuBz4PPDEiXh1Vx6ZXUoXY5+ZY40eA34qIF0bEirrOkyPi6My8DtgOnBcRB9Z7mb+1n/VtBt4WEU+t614TEa+o7/9aRDwjInqAX1IdN7y/XvdrImJNZk5RHQe+f4Z1f4Nq7/BPI6Inqk5bvwV8fC5/aN2J6rwmsw+ier53RcRR1F8AZvG0iHhZ/Zr+d6rX6+tzKKPp6z+TzHxxZj68ye3FM/1O/fx+st7O6dNaAKR5MWjV6T5P9WG353Ye8B6qTkM3U30wf7Fh+YcAb6baM/o58DzgTfW8XwO+ERF3AJ8F/jir3qW3AL9Jtad0C1UT829m5s1zKTAzf0LVgekc4CaqPdK38p//X68BnlWv+8+BT1CFSrP1fRp4B/Dxumn0SmBPIBxMtYd+K1WT7y3AO+t5rwN21L+zod7u9HXfQxWsL6Z6/t4P/LfM/N5c/laqvdB/bzLvfOBXgV3ANuBT+1nXZ6ia+fd0RHtZ/SVhf95D89e/VZ5N9Z44hSrQ9+wBPwf2DgJyR4HtagmKTC/8Li2mqE4r+l5mzron1mnq1oALM7PpqU/zWNd5wPGZ+doFFyZ1OPdopcLq5t7HR8RDIuJFVHu/F7W5rHnLzJ2tCFlpuXEUGqm8x1A1oz6SqmPNmzLz2+0tSdJiselYkqSCbDqWJKmgjmo6Puyww7K3t7fdZUiSNC+XXXbZzZl5+EzzOipoe3t72b59e7vLkCRpXiJi+uhye9l0LElSQQsO2noUnG9GdWmwqyLi/Hr6cVFdYuua+hJXBy68XEmSuksr9mjvBn49M0+kuorGiyLimVQj27w7M4+nGvllsAXbkiSpqyw4aLOyZyiynvqWVJfz+mQ9fSvVJbMkSVpWWnKMth5I/XLgRuDLwP8DbsvMe+tFdtLkkmMRcUZEbI+I7TfddFMrypEkqWO0JGgz877MPInqMldPB548+2/s87sXZOZAZg4cfviMPaMlSepaLe11nJm3AeNUVyo5pL78FVQB3IrrXUqS1FVa0ev48Ig4pL6/GngBMEkVuC+vFzud6pJYkiQtK60YsOIIYGtErKAK7gsz83MRcTXV9TT/HPg2sKUF25IkqassOGgz8zvAr8ww/Vqq47WSJC1bjgwlSVJBBq0kSQUZtJIkFWTQat7Gxsbo7+9nxYoV9Pf3MzY21u6SJKljddRl8tT5xsbGGB4eZsuWLaxdu5aJiQkGB6thrNevX9/m6iSp80RmtruGvQYGBtLr0Xa2/v5+TjvtNC666CImJyfp6+vb+/jKK69sd3mS1BYRcVlmDsw0zz1azcvVV1/NnXfe+YA92h07drS7NEnqSB6j1bwceOCBnHnmmaxbt46enh7WrVvHmWeeyYEHerlhabmxv8bcGLSal3vuuYfR0VHGx8eZmppifHyc0dFR7rnnnnaXJmkR7emvMTo6yl133cXo6CjDw8OG7Qw8Rqt58RitJKg+C0ZHR1m3bt3eaePj42zcuHFZfhbMdozWPVrNy/DwMB/72Mf2+Rb7sY99jOHh4XaXJmkRTU5Osnbt2n2mrV27lsnJyTZV1LnsDKV52XMKz8aNG/fu0Y6MjHhqj7TM9PX1MTExsc8e7cTEBH19fW2sqjMZtJq39evXG6zSMjc8PMzg4OADzkAYGRlpd2kdx6CVJM2brVtzZ2coSZIWyM5QkiS1iUErSVJBBq0kSQUZtJIkFWSvY81J79C2eS2/Y9NLC1UiSd3FoNWczBScvUPbDFRJ2g+bjiVJKsiglSSpIINWkqSCPEYrSZozO0bOn0ErSZozO0bOn03HkiQVZNBKklSQQStJUkEeo9U+Tjz/Ynbtnprz8nPtGLFmdQ9XnHvKgy1LkrqWQat97No9VaRTw3x7KkrSUmHTsSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBdnrWPs4qG+IE7YOFVgvgEO0SVp+DFrt4/bJTZ7eI0ktZNOxJEkFGbSSJBVk0EqSVJDHaCVJM5rP2OeOe96cQStJmlGJsc+XY8dIg1YPUOIfYc3qnpavU5K6gUGrfczn22vv0LYipwJJ0lJiZyhJkgoyaCVJKsiglSSpII/RSpJmVGLs8+U47rlBK0maUYmxz5fj6T02HUuSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBXkgBWSpKZaPcDEcrxkpkErSZrRXEeF8pKZszNoNSfNvtU2m+4/nSRVDFrNicEpSQ+OnaEkSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSpowUEbEcdExHhEXB0RV0XEH9fTD42IL0fED+ufj1h4uZIkdZdW7NHeC5yVmU8Bngn8YUQ8BRgC/jkznwD8c/1YkqRlZcFBm5nXZ+a36vu3A5PAUcCpwNZ6sa3AaQvdliRJ3aalFxWIiF7gV4BvAI/OzOvrWTcAj27yO2cAZwAce+yxrSxHktRiXslr/iIzW7OiiIcD/wqMZOanIuK2zDykYf6tmTnrcdqBgYHcvn17S+qRJGmxRMRlmTkw07yW9DqOiB7gH4CPZuan6sn/ERFH1POPAG5sxbYkSeomreh1HMAWYDIz39Uw67PA6fX904HPLHRbkiR1m1Yco/0vwOuA70bE5fW0c4BNwIURMQhcB/zXFmxLkqSusuCgzcwJIJrM/o2Frl+SpG7myFCSJBVk0EqSHpSxsTH6+/tZsWIF/f39jI2NtbukjtTS82glScvD2NgYw8PDbNmyhbVr1zIxMcHg4CAA69evb3N1naVl59G2gufRSlJ36O/vZ3R0lHXr1u2dNj4+zsaNG7nyyivbWFl7zHYerUErSZq3FStWcNddd9HT07N32tTUFKtWreK+++5rY2XtUXzACknS8tLX18fExMQ+0yYmJujr62tTRZ3LoJUkzdvw8DCDg4OMj48zNTXF+Pg4g4ODDA8Pt7u0jmNnKEnSvO3p8LRx40YmJyfp6+tjZGTEjlAz8BitJEkL5DFaSZLaxKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCjJoJUkqyKCVJKmgA9pdgNRqvUPb5rX8jk0vLVSJJBm0WoJmCs7eoW0GqqS2sOlYkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIAesUFc78fyL2bV7ak7LznXEqDWre7ji3FMWUpYk7WXQqqvt2j3V8hGf5juEoyTNxqZjSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIIRjV1Q7qG+KErUMtXidAa4d1lLR8GbTqardPbnKsY0kdzaZjSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKstexul6rewmvWd3T0vVJWt4MWnW1uZ7a0zu0reWnAUnSXNh0LElSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQS0J2oj4QETcGBFXNkw7NCK+HBE/rH8+ohXbkiSpm0RmLnwlEc8F7gA+lJn99bS/BH6emZsiYgh4RGaePdt6BgYGcvv27QuuR8vbfK9P6+XzJC1URFyWmQMzzWvJ9Wgz85KI6J02+VTg5Pr+VuArwKxBK7WCwSmpk5Q8RvvozLy+vn8D8OiZFoqIMyJie0Rsv+mmmwqWI0nS4luUzlBZtU/P2EadmRdk5kBmDhx++OGLUY4kSYumZND+R0QcAVD/vLHgtiRJ6kglg/azwOn1/dOBzxTcliRJHalVp/eMAV8DnhQROyNiENgEvCAifgg8v34sSdKy0qpex+ubzPqNVqxfkqRu5chQkiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklTQAe0uQJKkEnqHts1r+R2bXlqkDoNWkrQkNQvO3qFtxUJ1JjYdS5JUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJU0AHtLkCSpIU68fyL2bV7as7L9w5t2+8ya1b3cMW5pyykLMCglSQtAbt2T7Fj00tbus65hPFc2HQsSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFeR5tJKkrndQ3xAnbB1q8ToBFn5urkErSR1kbGyMkZERJicn6evrY3h4mPXr17e7rI53++Smjh2wwqCVpA4xNjbG8PAwW7ZsYe3atUxMTDA4OAhg2HYxj9FKUocYGRlhy5YtrFu3jp6eHtatW8eWLVsYGRlpd2laAINWkjrE5OQka9eu3Wfa2rVrmZycbFNFagWDVpI6RF9fHxMTE/tMm5iYoK+vr00VqRUMWknqEMPDwwwODjI+Ps7U1BTj4+MMDg4yPDzc7tK0AHaGkqQOsafD08aNG/f2Oh4ZGbEjVJczaCWpg6xfv95gXWKKNx1HxIsi4vsRcU1EtPZsYklaYjZu3MiqVauICFatWsXGjRvbXZIWqOgebUSsAN4HvADYCVwaEZ/NzKtLbleSutHGjRvZvHkz73jHO9iwYQObN2/m7LPPBmB0dLTN1XW+Vg0wscea1T0tWU9kZktWNOPKI54FnJeZL6wfvw0gM/9ipuWPO+64PPfcc4vVI0md7JJLLuFxj3scRx999N5pO3fu5Nprr+W5z31uGytbWr5+7S0883GPbOk63/CGN1yWmQMzzSt9jPYo4CcNj3cCz2hcICLOAM4AOOqoo+a84r+Kv2pBeTM7K88qtm6p0devvWVey7f6w6EZ/7/K2O/z+jy4lEv3nXZMdXvA9GmW8/PazGz/XzPNK/X/VXqP9uXAizLz9+vHrwOekZlnzrT8wMBAbt++vVg9UrfoHdrW8nFb1flWrVrF29/+dt785jfvnfaud72Lc845h7vuuquNlWl/IqJte7Q/pfo+tsfR9TRJ0jRvfOMb9x6TbTxGu2HDhjZXpoUoHbSXAk+IiOOoAvZVwKsLb1OSutKeDk/nnHMOZ511FitXrmTDhg12hOpyRYM2M++NiDOBLwErgA9k5lUltylJ3Wx0dNRgXWKKn0ebmZ/PzCdm5uMzc1EvQTE2NkZ/fz8rVqygv7+fsbGxxdy8JElLd2Qor+soSeoES/aiAl7XUZLUCZZs0HpdR0lSJ1iyQet1HSVJnWDJBq3XdZTKcvB7aW6WbGcor+solePg99LcFR2Ccb4cglGqdPoQjA4VKO1rtiEYl2zTsaRy7r777gcMC7hhwwbuvvvuNlUkda4l23QsdZoTz7+YXbun5rz8XK6tuWZ1D1ece8pCynpQVq5cyebNm/fZo928eTMrV65c9FqkTmfQSotk1+6pljcHt/pC13Pl4PfS3Bm0kubNwe+luTNoJT0oDn4vzY2doSRJKsiglSSpIINWkqSCDFpJkgqyM5S0SA7qG+KErUMtXidA544gJcmglRbN7ZOblsx5tJLmzqZjSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKcmQoaRG1eiSnNat7Wro+Sa1n0EqLZD7DL/YObWv5cI2S2sOmY0mSCjJoJUkqyKCVJKkgg1aSpIIMWkmSCrLXcRvN51QPe6BKUncyaNtopvD0tA5JWlpsOpYkqSCDVpKkggxaSZIKMmglSSrIzlCSliR79atTGLSSliR79atT2HQsSVJBBq0kSQUZtJIkFeQxWqmNZuuwM9M8jy9K3cegXQQnnn8xu3ZPzXn5ufaWXLO6hyvOPeXBlqUOYHBKS59Buwh27Z4q8oE6n9MXJEnt4TFaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrI03sWwUF9Q5ywdajAegE8D1OSOplBuwhun9zkebSStEzZdCxJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQA1YskhKDS6xZ3dPydUqSWsugXQTzGRWqd2hbkVGkJEntYdBK6monnn8xu3ZPzXn5ubYurVndwxXnnvJgy5L2MmgldbVdu6ccS1wdzc5QkiQVZNBKklSQQStJUkEGrSRJBS2oM1REvAI4D+gDnp6Z2xvmvQ0YBO4D/igzv7SQbUnSTA7qG+KErUMF1gvgqXZauIX2Or4SeBnwt40TI+IpwKuApwJHAv8UEU/MzPsWuD1J2sftk5vsdayOtqCm48yczMzvzzDrVODjmXl3Zv4IuAZ4+kK2JUlSNyp1Hu1RwNcbHu+spz1ARJwBnAFw7LHHFiqnMzX7xjzTdEeLkqTutN+gjYh/Ah4zw6zhzPzMQgvIzAuACwAGBgZyoevrJoanJC19+w3azHz+g1jvT4FjGh4fXU+TJGlZKXV6z2eBV0XEyog4DngC8M1C25IkqWMtKGgj4nciYifwLGBbRHwJIDOvAi4Erga+CPyhPY4lScvRgjpDZeangU83mTcCjCxk/ZIkdTtHhpIkqSCDVpKkggxaSZIKMmglSSqo1MhQkrRoSoxLvGZ1T8vXqeXJoJXU1eYzwlrv0DZHZNOis+lYkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDmh3AZJUQu/QtjlP37HppaXL0TJm0EpakgxPdQqbjiVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWkqSCIjPbXcNeEXETcF2BVR8G3FxgvSVYaxndVCt0V73WWoa1llOi3sdm5uEzzeiooC0lIrZn5kC765gLay2jm2qF7qrXWsuw1nIWu16bjiVJKsiglSSpoOUStBe0u4B5sNYyuqlW6K56rbUMay1nUetdFsdoJUlql+WyRytJUlsYtJIkFbQkgjYiTouIjIgnN0x7ekR8JSJ+GBHfiohtEXFCPe+8iPhpRFzecDtkkWodjoirIuI79XafUdf5/YZaPlkv+zcR8T+n/e77FqPOenv31fVcUT+Hz66n99bP9583LHtYRExFxHvrx+dFxFsWq9Z51L27nnd1RGyOiLb/DzR5T5wZEdfUz/Nh7a5xjya1frR+/14ZER+IiJ521wlNa91Svy++ExGfjIiHt7vOPWaqt2He30TEHe2sr1GT5/aDEfGjhs+xk9pdJzStNSJiJCJ+EBGTEfFHRYvIzK6/AZ8A/g04v378aGAH8OyGZdYCp9X3zwPe0oY6nwV8DVhZPz4MOBL4CjAww/IHA9cCj6tvPwIOWcR672i4/0LgX+v7vXVd326Y/ybgcuC97XyO51D3lfX9A4BLgJe1o8Y5vCd+pa53B3BYO2ucQ60vAaK+jQFv6uBaD25Y5l3AULtrna3e+v4A8OHG93Un1gp8EHh5u+ubY61vAD4EPKSe/qiSdRxAl6u/ka4F1gH/CJwLnAlszcyv7lkuMyfaU+E+jgBuzsy7ATLzZoCImHHhzPxFRAwD760n/c/MvG0R6pzJwcCtDY/vBCYjYiAztwOvBC6kehN3kul1A5CZ90bEV4HjF7+kfcz4ngB+Bs3fG20ya60AEfFN4Og21DZds1oBiOqJXQ10Sm/QZp8NK4B3Aq8Gfqd95e1jXp9jbdas1jcBr87M++vpN5Ysou3NZi1wKvDFzPwBcEtEPA14KvCt/fzenzQ0cYwXr7JyMXBM3Vzx/oh4XsO8jzbU8849EzNzDHgE1TfxDy9SnXusruv5HvD3wP+aNv/jwKsi4hjgPho+cNtsf3UTEQ8FfgP47mIXN81s74lOM2utdZPx64AvtqW6fTWtNSL+D3AD8GRgtF0FTtOs3jOBz2bm9W2sbbrZ3gcjdRPtuyNiZbsKbNCs1scDr4yI7RHxhYh4QskilkLQrqf6wKf+uX76AhHxjbod/q8bJr87M0+qb+sWo9DMvAN4GnAGcBPwiYh4fT37NQ31vLWh9qOpvpUd2YbjSbvrep4MvAj4UOz7tfWLwAuAV1E133eK2ep+fERcDvw7sC0zv9CuImG/74mOModa3w9ckpn/1oby9jFbrZn5BqqWl0mqlpi2a1LvOcAr6JwvA8Csz+3bqL68/BpwKHB2u2rcY5ZaVwJ3ZTUM498BHyhdSNfeqF7MO6kuRLAD+AnwY6o9mPOnLfty4IP1/fNo0/HDGWr6R5oco62X+QfgdGAT8M5Fru+OaY//A3gU+x7r/ADV3sGhwOvpsGO0zeru1Nue90TD4x10yDHa2WqlOmRzEfUxr067TX9e62nPBT7X7tpmqXeq/t/aUd/uB65pd21zfG5P7sTntuEz93vAcfW0AHaV3G6379G+HPhwZj42M3sz8xiqDkNfBl4fdW/T2kPbUmGDiHjStCaKk5jlakUR8WKqgPgQ1ZeHl0XEU4oW2byWJwMrgFumzfor4OzM/PniV7V/s9TdEeb7nminZrVGxO9TdTpbn/Uxr3ZrUuuPI+L4en4Av031gdt2Ter928x8TP3Z1gvcmZnt7lMw2/vgiHp+AKcBVy5+dfua5f/rIqp+PQDPA35Qso5u7wy1HnjHtGn/UE9/JfCOiDgKuJHqkkh/1rDcn0TEaxsen5aZOwrWCvBwYDSqU4nuBa6hatL4JNUx2t31cjcDvwm8h6oXXwK/jIi3UnWM+vXCde6xum5mhepb3+mZeV9j63FmXgVctUj1zNV+6+4gM74n6tMN/hR4DPCdiPh8Zv5++8oEmr9/b6D68Ppa/Rx/KjP/rNlKFslMtW4APh0RB1O9L66g6i3fCZo9t52oWa0XRsThVM/t5VTPd7s1q/Veqs/cPwHuAIr+bzkEoyRJBXV707EkSR3NoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkgv4/R91zFHRkUJYAAAAASUVORK5CYII=)
%% Cell type:markdown id: tags:
### Multicollinearity
You will notice that several of the regression coefficient distributions differ significantly between the original linear model and the models regularised using Lasso or Ridge regression. The key examples here are `S1` to `S4`; in particular observe that S3 has gone from a positive effect to a negative effect which S2 and S4 have little to no effect.
We can look back at the correlation table to explain why this change occurs: it is due to multicollinearity. The correlation table shows that some of our input variables (`S1` to `S4`) have relatively strong correlations with one another. When we fit a linear model to input data containing two correlated variables A and B, we can produce multiple models with equivalent error by increasing the coefficient of A, while decreasing the coefficient of B to counteract the effect. Introducing regularisation tends to reduce this effect by introducing a penalty for the training algorithm's tendency to increase the coefficient values. The results of these regularised models indicate that we could potentially remove `S2` and `S4` from the model; they are strongly correlated with `S1` and `S3` respectively so do not contribute useful information.
%% Cell type:markdown id: tags:
### Feature Selection
These results indicate that we could remove some unimportant/confounding features from the model in order to reduce variance. We have identified here that BMI, BP and S5 are the features with the highest importance. It may be worth investigating whether we can achieve good predictions on the test set by using a model which only uses these features.
Interestingly, this does align with what we saw in the correlation coefficients - BMI, BP and S5 do have the strongest correlations with Y.
Including 'confounding' features in a model can be problematic. They may result in overfitting to the training data, and lead us to observe an effect that is actually not there. They can also inflate the importance of other variables in the case of multi-collinearty.
This, it is important to conduct some trials of models with different subsets of features. Correlation tables work well as a guide here, but ultimately we need to assess the effect of model changes on the error measure for the **test** set.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment