01-ML_Workflow_Diabetes_Fin.ipynb 198 KB
Newer Older
Simon Clarke's avatar
Simon Clarke committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression Workflow (Diabetes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lesson we will go over the workflow for regression modelling. This investigates the effect of various features on the tendency of patients to develop diabetes, using the `sklearn` methods `LinearRegression` and `Ridge` regularization. We will look at how coefficients can vary signficantly due to overfitting, and how that can be alleviated using regularization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Modules and Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First import the Diabetes dataset. The same dataset can be imported from the `sklearn` example datasets, but is already normalized. We will use the unnormalized dataset initially."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split # for splitting the data into training and testing sets\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso # models we are going to use\n",
    "from sklearn.metrics import r2_score # for comparing the predicted and test values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('Diabetes_Data.csv') # read the Diabetes dataset in to a pandas dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`BP` is blood pressure, `BMI` is body mass index (healthy range for adults is 18.5 to 25) and `S1`-`S6` are various blood serum measurements. `Y` is the target variable, and is a measure of disease progression one year after the original measurements. Aim is to predict `Y` from the other variables.\n",
    "\n",
    "We can look at the first few values of the variables, and at the descriptive statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>SEX</th>\n",
       "      <th>BMI</th>\n",
       "      <th>BP</th>\n",
       "      <th>S1</th>\n",
       "      <th>S2</th>\n",
       "      <th>S3</th>\n",
       "      <th>S4</th>\n",
       "      <th>S5</th>\n",
       "      <th>S6</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>59</td>\n",
       "      <td>2</td>\n",
       "      <td>32.1</td>\n",
       "      <td>101.0</td>\n",
       "      <td>157</td>\n",
       "      <td>93.2</td>\n",
       "      <td>38.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.8598</td>\n",
       "      <td>87</td>\n",
       "      <td>151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>48</td>\n",
       "      <td>1</td>\n",
       "      <td>21.6</td>\n",
       "      <td>87.0</td>\n",
       "      <td>183</td>\n",
       "      <td>103.2</td>\n",
       "      <td>70.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.8918</td>\n",
       "      <td>69</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>72</td>\n",
       "      <td>2</td>\n",
       "      <td>30.5</td>\n",
       "      <td>93.0</td>\n",
       "      <td>156</td>\n",
       "      <td>93.6</td>\n",
       "      <td>41.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.6728</td>\n",
       "      <td>85</td>\n",
       "      <td>141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>25.3</td>\n",
       "      <td>84.0</td>\n",
       "      <td>198</td>\n",
       "      <td>131.4</td>\n",
       "      <td>40.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.8903</td>\n",
       "      <td>89</td>\n",
       "      <td>206</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>23.0</td>\n",
       "      <td>101.0</td>\n",
       "      <td>192</td>\n",
       "      <td>125.4</td>\n",
       "      <td>52.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.2905</td>\n",
       "      <td>80</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   AGE  SEX   BMI     BP   S1     S2    S3   S4      S5  S6    Y\n",
       "0   59    2  32.1  101.0  157   93.2  38.0  4.0  4.8598  87  151\n",
       "1   48    1  21.6   87.0  183  103.2  70.0  3.0  3.8918  69   75\n",
       "2   72    2  30.5   93.0  156   93.6  41.0  4.0  4.6728  85  141\n",
       "3   24    1  25.3   84.0  198  131.4  40.0  5.0  4.8903  89  206\n",
       "4   50    1  23.0  101.0  192  125.4  52.0  4.0  4.2905  80  135"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>SEX</th>\n",
       "      <th>BMI</th>\n",
       "      <th>BP</th>\n",
       "      <th>S1</th>\n",
       "      <th>S2</th>\n",
       "      <th>S3</th>\n",
       "      <th>S4</th>\n",
       "      <th>S5</th>\n",
       "      <th>S6</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "      <td>442.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>48.518100</td>\n",
       "      <td>1.468326</td>\n",
       "      <td>26.375792</td>\n",
       "      <td>94.647014</td>\n",
       "      <td>189.140271</td>\n",
       "      <td>115.439140</td>\n",
       "      <td>49.788462</td>\n",
       "      <td>4.070249</td>\n",
       "      <td>4.641411</td>\n",
       "      <td>91.260181</td>\n",
       "      <td>152.133484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>13.109028</td>\n",
       "      <td>0.499561</td>\n",
       "      <td>4.418122</td>\n",
       "      <td>13.831283</td>\n",
       "      <td>34.608052</td>\n",
       "      <td>30.413081</td>\n",
       "      <td>12.934202</td>\n",
       "      <td>1.290450</td>\n",
       "      <td>0.522391</td>\n",
       "      <td>11.496335</td>\n",
       "      <td>77.093005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>97.000000</td>\n",
       "      <td>41.600000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.258100</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>25.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>38.250000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>23.200000</td>\n",
       "      <td>84.000000</td>\n",
       "      <td>164.250000</td>\n",
       "      <td>96.050000</td>\n",
       "      <td>40.250000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.276700</td>\n",
       "      <td>83.250000</td>\n",
       "      <td>87.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>25.700000</td>\n",
       "      <td>93.000000</td>\n",
       "      <td>186.000000</td>\n",
       "      <td>113.000000</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.620050</td>\n",
       "      <td>91.000000</td>\n",
       "      <td>140.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>29.275000</td>\n",
       "      <td>105.000000</td>\n",
       "      <td>209.750000</td>\n",
       "      <td>134.500000</td>\n",
       "      <td>57.750000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>4.997200</td>\n",
       "      <td>98.000000</td>\n",
       "      <td>211.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>42.200000</td>\n",
       "      <td>133.000000</td>\n",
       "      <td>301.000000</td>\n",
       "      <td>242.400000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>9.090000</td>\n",
       "      <td>6.107000</td>\n",
       "      <td>124.000000</td>\n",
       "      <td>346.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              AGE         SEX         BMI          BP          S1          S2  \\\n",
       "count  442.000000  442.000000  442.000000  442.000000  442.000000  442.000000   \n",
       "mean    48.518100    1.468326   26.375792   94.647014  189.140271  115.439140   \n",
       "std     13.109028    0.499561    4.418122   13.831283   34.608052   30.413081   \n",
       "min     19.000000    1.000000   18.000000   62.000000   97.000000   41.600000   \n",
       "25%     38.250000    1.000000   23.200000   84.000000  164.250000   96.050000   \n",
       "50%     50.000000    1.000000   25.700000   93.000000  186.000000  113.000000   \n",
       "75%     59.000000    2.000000   29.275000  105.000000  209.750000  134.500000   \n",
       "max     79.000000    2.000000   42.200000  133.000000  301.000000  242.400000   \n",
       "\n",
       "               S3          S4          S5          S6           Y  \n",
       "count  442.000000  442.000000  442.000000  442.000000  442.000000  \n",
       "mean    49.788462    4.070249    4.641411   91.260181  152.133484  \n",
       "std     12.934202    1.290450    0.522391   11.496335   77.093005  \n",
       "min     22.000000    2.000000    3.258100   58.000000   25.000000  \n",
       "25%     40.250000    3.000000    4.276700   83.250000   87.000000  \n",
       "50%     48.000000    4.000000    4.620050   91.000000  140.500000  \n",
       "75%     57.750000    5.000000    4.997200   98.000000  211.500000  \n",
       "max     99.000000    9.090000    6.107000  124.000000  346.000000  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the maximum and minumum values of all features are similar, that suggests there are no missing data points and we don't need to clean the data.\n",
    "\n",
    "Here the field `SEX` only has two values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 1])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['SEX'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Categorical data such as this should be converted to binary columuns using the `sklearn` method `OneHotEncoder`. However, since `SEX` is already binary, there is no need to do this. If we had a field which had three categories, for example, the data came from three different states, then we could create a binary field for each state, with the value 1 if the person is resident in the state and 0 otherwise. For more details see `sklearn.feature_extraction.DictVectorizer` and `sklearn.preprocessing.OneHotEncoder`.\n",
    "\n",
    "We can look at the variable correlations to search for patterns. `SEX` doesn't seem to be important in predicting `Y`, and `AGE`, `S1` and `S2` are only marginally important. The most important variables seem to be `BMI`, `BP`, `S4` and `S5`. There is a strong correlation between `S1` and `S2`, and `S3` and `S4`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAHSCAYAAAA+DMuQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1R7G8e/ZBKQjNaFJEUE6XqrSBZSuAoqoCDawgIgFpYmIioKiIoKCImADpChVelWqIr2FHtLovWVz7h8JIT2B7GZX9/3cZ59nZ+Zs5t0h3v3llFljrUVERETEUxyeDiAiIiK+TcWIiIiIeJSKEREREfEoFSMiIiLiUSpGRERExKNUjIiIiIhH+WfAObR2WERE/u1Mhp5t2zT3fHZWaJex7yON1DMiIiIiHqViRERERDxKxYiIiIh4VEbMGREREZEbYJ1Ot/xcr5wwgnpGRERExMPUMyIiIuJtnJGeTpChVIyIiIh4GRvlnmJEwzQiIiIiSVDPiIiIiLdx0wRWb6WeEREREfEo9YyIiIh4GasJrCIiIuJRPlaMaJhGREREPErFiIiIiHiUihERERHxKM0ZERER8TLuuumZt1IxIiIi4m10n5Foxpg74zy/JcGx2u4MJSIiIr4jpZ6Rn4D/xTxfHec5wKgE2yIiIuIivnafkZQmsJpknie1Hf+gMV2NMRuMMRvGjBlz0+FERETkvy+lnhGbzPOktuMftHYMMCYtbUVERCQBD/WMGGOaAZ8DfsA31toPExwvDowDCgAngCestcHpPW9KxUhRY8wIontBrj0nZrtIek8sIiIi3sMY4wd8CTQFgoH1xpiZ1trtcZp9DEy01k4wxtwLDAE6pffcKRUjb8R5viHBsYTbIiIi4iI2yiOraWoCQdbafQDGmEnAA0DcYqQ80Cvm+VLgV1ecONlixFo7wRUnEBERkRvjoQmsRYDDcbaDgVoJ2mwC2hE9lPMQkNMYk89aezw9J05paW9dY8yTcbanGmOWxDzuTc9JRUREJOPFXWAS8+ga93ASL0k47/N1oIExZiPQADgCpLtySmmYZhDQI852WaALkB3oCyxJ78lFREQkCW7qGUmwwCShYKBYnO2iQEiC14cAbQGMMTmAdtba0+nNldLS3lwJJq3ssdb+Za1dAeRM74lFRETEq6wH7jDGlDTGZAYeBWbGbWCMyW+MuVY79CF6ZU26pVSM3Bp3w1rbNs5mgCtOLiIiIt7BWhsJdAfmAzuAKdbabcaYd40xbWKaNQR2GWN2E10LvO+Kc6c0TLPTGNPSWjsn7k5jTCtglytOLiIiIol5aDUN1tq5wNwE+96O83wqMNXV502pGOkFzDHGtAf+jtlXDbgHaOXqICIiIuKbUlraG2SMqQw8DlQgekbtcuAboCfwUoYkFBER8TU+9t00KfWMYK29DIwzxtwFdAQGAvuBaRmQTURERHxAssWIMaYM0TNpOwLHgcmAsdY2yqBsIiIiPsnXvrU3xQmswEqgtbU2CMAY0yuF9iIiIuIKPlaMpLS0tx0QBiw1xow1xjQm6buziYiIiNy0lCawzgBmGGOyAw8SvbomwBgzGphhrV2QQRlFRER8iqeW9npKSj0jAFhrz1trf7TWtiL61rD/AG+5PZmIiIj4hBRX0yRkrT0BfB3zEBEREXfwsTkjN1SMiIiIiPtZp28N02RIMdKkdIWMOM0NWxS0zdMRREREfF6qc0ZERERE3EnFiIiIiHiU5oyIiIh4GV+7A6t6RkRERMSj1DMiIiLibaJ8q2dExYiIiIiX8bWlvRqmEREREY9Sz4iIiIi3Uc+IiIiISMZRz4iIiIiX8bWlvSpGREREvI2GaUREREQyjnpGREREvIyW9oqIiIhkIK8pRmrUr8t3C2YzYfE8Hu32bKLjlWpUY/RvvzB/5ybqNbsvdn+V2jX5aua02MfcbX9zT5N7MzK6iIiIpINXDNM4HA56vNOPNzs/x9GwcL6cPpk/Fy/lUNDe2DYRIaEM7d2PR57tEu+1m9as4/k27QDImTs3ExbP469Vf2ZkfBEREZeyUb41TOMVxUjZKpUIOXiY0MPBACybM5c6TRrFK0bCj4QAEBVlk/059Zvdx/rlK7l86ZJ7A4uIiIjLpDhMY4z5whiTM4n9dxpjFrkqRP6AACJCQ2O3j4aFky8g4IZ/TsNWzVkye66rYomIiHiG0+meh5dKbc5IGPCPMeYxAGNMNmPMUGAm8GVyLzLGdDXGbDDGbBgzZkyqIYxJYqdNvgckKXkL5Kdk2TvYsPKPG3qdiIiIeFaKwzTW2veNMT8BI40xzwOFgSlAVWvthRReNwa4VoXYKUM/TzHE0bBwChYqFLtdIDCA4xERaXsHMRq0aMYfCxbjjPStu9aJiMh/j5b2Jnati8I/pv2OlAqRm7Fr81aKFL+NwKJF8M+UiYYtW/Dn4qU39DPubd1CQzQiIiL/QqnNGekPLAImWmvvAeoBDxhjlhtjyrsqRJTTyReD3ufD78Ywbv5Mls/9nYN79tK5Z3fubtwIgLKVKvLzqsXUb34fvQYP5Jt5v8W+PqBIYQoEBrJ57XpXRRIREfEY64xyy8NbGZvC3AxjzOdAf2vt2QT7mwPDrbXl0nAO26R0hfSldJNFQds8HUFERP4dkprd6DYn+je4sYmTaZT3veUZ+j7SKsWeEWttz4SFSMz+eUBVt6USERERn5HaMM2UOM8/SnB4llsSiYiI+DjrdLrl4a1Sm8B6R5znTRMcK+DiLCIiIuKDUrsDa0pjVm4ZzxIREfF11ulbH7GpFSPZjDF3Ed2DktUY87+Y/QbI6tZkIiIi4hNSK0ZCgU+ILj7CgI/jHAtzVygRERHxHakVI28Ch621oQDGmM5AO+AA8I5bk4mIiPgob74niDukNoH1K+AygDGmPjAEmACc5vrt3kVERERuWmo9I37W2hMxzzsAY6y104Bpxph/3BtNRETEN/laz0iqxYgxxt9aGwk0BrrewGtFRETkJtgoraaJ62dguTHmGHARWAlgjClN9FCNiIiISLqkWIxYa983xiwGCgEL7PUvsnEAPdwdTkRExBfpPiMJWGvXJLFvt3viiIiIiK/RvA8REREvY733a2TcIkOKkUVB2zLiNCIiIvIvpJ4RERERL6M5I27Q/I5KGXGaGzZvzxYqlyjl6RjJ2nxgn6cjiIiIB0T51m1GUr0Dq4iIiIhbqRgRERERj1IxIiIiIh6lCawiIiJeRkt7RURExKN8rRjRMI2IiIh4lHpGREREvIyW9oqIiIhkIPWMiIiIeBlfmzOiYkRERMTLREUZT0fIUBqmEREREY9Sz4iIiIiX0QRWERERkQzkNcVItXp1GDt/Jt8umsPDXZ9JdLxijWp88etkZu/YSN1mTeMde/qNXoyeM53Rc6ZTv8X9LstUp0F9Zi5exOxlS3j6hecTHc+UOTNDR45g9rIl/PjrdAoXLQKAv78/730yjGm/z+PXRQt45sUXYl/zxDNPM33B70yfP4+PRnxO5lsyuyyviIjIv5FXFCMOh4OX3unHgGdfpFvzB2jYqjm3lS4Vr01ESCifvDmApbPmxttfo2E9bq9QjpfaPMwr7R+n3bNPkS1Hdpdk6vvuIF7o8hQPNr2f5m1aU6p06Xht2j7yCGdOn6FVw3v5/ttxvPLWmwDc16IFmTJnpl2z5jzaqg3tH+tI4aJFKBgQwONdOtOx9QO0vb85DoeDZq1bpzuriIjIv5lXFCNlKlci5OAhwg4HE3k1kuVz5lG7caN4bSKOhHBg126stfH231b6dras20CU08nlixfZv3MX1erVTXemilWrcOjgQY4cPkzk1av8Pms2je6L3yPT8L4mzJw2DYCFc+dR6557ALBYsmXNhp+fH7dkycLVK1c5d/YcQOw+Pz8/smTNytHw8HRnFRGR/xbrdM8jNcaYZsaYXcaYIGPMW8m0ecQYs90Ys80Y85Mr3m+KE1iNMa+mdNxaO9wVIfIHFuRoaFjs9rGwcMpWqZym1+7fuYvHur/AjHETuSVrFirXrsmhoL3pzhQQEEh4SGjsdnhoKJWqVk3QJiC2jdPp5NzZs9yaJw8L586jYdMmLF63hqxZszJ08HucOX2aM6dPM2HsNyz4cxWXLl1i9cpVrF65Kt1ZRUTkv8UTS3uNMX7Al0BTIBhYb4yZaa3dHqfNHUAfoI619qQxpqArzp3aapqcrjhJ6pK46Al6QJLz96rVlKlUkU+mfM/pEyfZuXETTqcL7haTZKQEmUziRtZaKlapQpQziia17iZX7tyMnzKZNav+4Mzp0zRq2oTm9Rpw9swZPh41kpYPPsCcX39Lf14REZH0qQkEWWv3ARhjJgEPANvjtHkO+NJaexLAWhvhihOnWIxYawfdzA81xnQFugJ8/fXXqbY/FhZOgUKBsdv5AwM4HpH29zdp9FgmjR4LQO/hHxFy4NANJk4sPCyMgMKFYrcDChXiaIJM19qEh4Xh5+dHjpw5OX3qFC0eaMMfy5cTGRnJiePH2fjXX1SoXAlrLcGHgzl54gQAi3+fT9Vq1VSMiIhIPFGeuQNrEeBwnO1goFaCNmUAjDF/AH7AO9ba39N74hTnjBhjRqT0SO511tox1trq1trqXbt2TTXE7i1bKVyiOAFFi+CfyZ8GLZuzZvGytL0Bh4Oct+YGoETZMpQsewd/rfozTa9NybZNmyleogRFihbFP1MmmrVuxbKFi+K1WbZwMW3atQOgaYvmrPtzNQChISHUjJk/kjVrVirfVZX9e/cRFhJC5buqkiVLFgBq1bmHfUFB6c4qIiKSFsaYrsaYDXEecT+kkxobSjhM4Q/cATQEOgLfGGNuTW+u1IZpnge2AlOAEJIOmm5RTiejB33Ae+O+ws/PjwVTZ3AoaC+der7E7i3bWLtkGWUqVWDAqM/JkSsntRo14ImXX+T5Fg/h5+/Pxz9PAODCuXMMe70PUS4YpnE6nXzw9juMnjgBPz8Hv075hb179vBir1fYvmULyxYtZsaUyXwwfDizly3h9KnT9O7xMgCTJn7P4GFDmb7gd4wx/PbLVPbs3AnAonm/M3nOLJyRkezYtp2pP09Kd1YREflvcdecEWvtGGBMMoeDgWJxtosS/dmfsM0aa+1VYL8xZhfRxcn69OQyieZBxD1oTD7gYaADEAlMBqZdGytKI9v8jkrpyeg28/ZsoXKJUqk39JDNB/Z5OoKIiETL0BmlW5rVTtvEyRtU6fc1yb4PY4w/sBtoDBwhusB4zFq7LU6bZkBHa21nY0x+YCNQ1Vp7PD25UhymsdYet9Z+Za1tBHQBbgW2GWM6peekIiIi4l2stZFAd2A+sAOYYq3dZox51xjTJqbZfOC4MWY7sBR4I72FCKTxu2mMMf8jemyoKTAP+Cu9JxYREZGkeeq7aay1c4G5Cfa9Hee5BV6NebhMavcZGQS0IrpCmgT0iamcRERERFwitZ6RAcA+oErM4wMTfW8NQ3SBlLY7k4mIiEiaeeKmZ56UWjFSMkNSiIiIiM9K7aZnBxPui5k9e9ymtAxHREREJI1Su+lZbWPMMmPMdGPMXcaYrUTfdyQ8ZnmPiIiIuFhUlHHLw1ulNkwzEugL5AaWAM2ttWuMMXcCPwPpvgWsiIiI+LbUihF/a+0CAGPMu9baNQDW2p0miS+JExERkfRzenEvhjukOEwDxF3pfDHBMc0ZERERkXRLrWekijHmDNFLebPGPCdmO4tbk4mIiPgob57f4Q6prabxy6ggIiIiEi3K+lYxktowjYiIiIhbpem7aURERCTjeOq7aTxFPSMiIiLiURnSMzJvz5aMOM1N2Xxgn6cjiIiIxOP0sTkjGVKM9K/cKCNOc8Pe27yUx8vV8HSMZP24Yz2fV2nq6RhJ6rlpoacjiIj8Z/naahoN04iIiIhHqRgRERERj1IxIiIiIh6lpb0iIiJeRhNYRURExKN0B1YRERGRDKSeERERES/ja8M06hkRERERj1LPiIiIiJdxWk8nyFjqGRERERGPUs+IiIiIl/G11TQqRkRERLyMJrCKiIiIZCD1jIiIiHgZTWD1oDvq1KDnzAn0mv0D9Z/umOh4jYdb033at7w0ZSzPjR9BgVLF4x3PHViQAWvmUqfzIy7PVrnu3QybO5VPfp9O62c7JzrevPNjDJ01mSG//kSfcaPIXzgw3vGs2bPzxbI5dO7/hsuzJVT8nuo8+ds4Os8aT/WnOyTbrnSTevTctJCC5cu4PZOIiEhyvKYYMQ4Hrfv2ZOILbzHiwS5Uat44UbGxee5iRrZ7hi8feY6V4yfR/I0X4x1v0fsl9qxa65ZsXQb0ZmjXnvRu/Qh3t7yPIreXjNfm4I5d9H/4Sfo8+BjrFiym4+svxzve/uXn2bn+b5dnSyprw749+PXFvnz/0LOUadaIvKVuS9QuU7asVH3sQUI373B7JhERkZSkWowYYx40xrxujLnfnUGKVryT44dCOHkkFGdkJFt+X0K5RnXitbl8/kLs88xZs4C93o9VrlEdTgSHELH3gMuz3V65AuGHDnM0+AjOq5GsmbuQavc2iNdm+7q/uHLpMgBBm7aQN6Bg7LES5e8kd/68bPnD9YVSQgEVy3L6cAhnjoQRFRnJ7t+XUarhPYna3f1SF/4aPwXn5StuzyQiIjfGiXHLw1ulWIwYY0YBvYB8wGBjzAB3BckVkJ/T4RGx22fCj5KrYP5E7Wp1eJBX5/zA/b26MefDLwDIlDUL9Z7uyNLRE9ySLW/BAhwPC4/dPhEeTp6AAsm2b9juATat/BMAYwyPv/kKPw0b4ZZsCeUomJ+zYUdjt89FHCNHQPzrWODO28kZWID9K9xfHImIiKQmtZ6R+sC91to+QEPgwbT8UGNMV2PMBmPMhjFjxqQxSuKKzdrEM3jWTv6V4S2fYP5nY2jYtRMAjV/swp/fT+XKxUtpPNcNMmnLBlCndXNKVSzH7G+/B6BJx/ZsWvEHJ+IUM26VWlZjqP/6C6z45OuMySMiIjfMad3z8Fapraa5Yq11AlhrLxiTxCddEqy1Y4BrVYjtP/LnVF9zJvwoueMMbeQKKMDZo8eTbb9l3hLa9HsFgKKVylGhSQPu79WNLDlzYG0UkZevsHbSr2mJm6oT4RHkCwyI3c4bEMCpiGOJ2lW4uyYPdHuK957sRuTVqwDcUbUyZatVpUnH9mTJlg3/TP5cunCRycNHuiRbQufCj5Iz8HqvTY6C+Tkfcf06Zs6elXylS9D+m48ByJY/L60/f5dZPd8mYvtut2QSERFJSWrFyJ3GmM0xzw1we8y2Aay1trKrghzZtpN8xYuQp0ggZ8KPUanZvfzy1nvx2uS7rQjHDx0BoEz92rHPv+nSM7bNvS905vKFiy4rRAD2bdlOYPHbKFCkMCciIqjdoilfvhF/xKp4uTI8804fPur6MmdOnIzdP6r39Xb1H2xFyYrl3FaIAIRv28WttxUhV5FAzoUfo0yzhvzeZ0js8SvnLjCmYfvY7XbffMzK4WNUiIiIeBGnpwNksNSKkXIZkgKIckYx+4MRdB49FIefg79+nUfE3gM0fvEpjmzfxc5lf1Kr40PcXqsaUZGRXDxzlmn9P8ygbE7GvzeUN78ZgcPhx/LpMzkStI92Pbqxf+sO/l66gsfe6EmWbFnp+Wl0pmOhYQx/6bUMyReXdUaxbMhIHhw9BONwsP3X+ZzYe5DaL3YmfNtu9i9fneGZRETkxvhaMWKSm/uQ7AuMyQ8ct2l/oe1fudENB8sI721eyuPlang6RrJ+3LGez6s09XSMJPXctNDTEUREMlKGLkUZX7WpW2Z4dPlnoVcuqUltNU1tY8wyY8x0Y8xdxpitwFYg3BjTLGMiioiI+BZfW9qb2jDNSKAvkBtYAjS31q4xxtwJ/Az87uZ8IiIi8h+XWjHib61dAGCMeddauwbAWrszjQtrRERE5AY5b3AKxb9davcZiYrz/GKCY751pURERMQtUusZqWKMOUP0xJ2sMc+J2c7i1mQiIiI+ytdW06RYjFhr/TIqiIiIiPgmr/nWXhEREfFNqQ3TiIiISAbztWEa9YyIiIiIR6lnRERExMv4Ws+IihEREREv4/Sxu2domEZEREQ8Sj0jIiIiXsbXhmnUMyIiIiIeZaz773/vWwNfIiLyX5ShX8j2YeXGbvnsfGvzYq/8YjkN04iIiHgZXxumyZBipEv5Whlxmhs2fvtaShcv7ukYyQo6eJBO5Wt6OkaSvt++DoDHy9XwcJKk/bhjvacjiIhIGqlnRERExMtoaa+IiIhIBlIxIiIiIh6lYRoREREvo2EaERERkQyknhEREREv42tLe9UzIiIiIh6lnhEREREv43T/3dG9inpGRERExKPUMyIiIuJlfG01jYoRERERL+NrxYiGaURERAQAY0wzY8wuY0yQMeatJI4/b4zZYoz5xxizyhhT3hXnVc+IiIiIl4nywARWY4wf8CXQFAgG1htjZlprt8dp9pO19quY9m2A4UCz9J7ba4qRSnVr81ifV3H4OVgxdSZzvpkY7/j9nTtSv/0DREVGcvbkKb7t/x7HQ8IAyFsogKff7UfewIJY4NNuvTgWEpruTPUbNKD/wIH4+fkxZdIkvh49Ot7xzJkzM2z4cCpWqsTJkyfp2b07R4KDqVylCu8NGQKAMYYRn33GwvnzAVi2ahXnz5/H6XTidDp5qHXrdOeE6OvXqc9rOPwcLJv6G7MTXL9mnR+jYfs2OCOdnD15irH9B8devwlbVnN4z14AjoeE8Wn3112S6ZrKde+mU9/XcDiis836ZkK84807P0aj9g/gdDo5c+IUY/u/y7GYbABZs2dn6JwpbFi0jAnvDXNpNhERiVUTCLLW7gMwxkwCHgBiixFr7Zk47bODa8aTvKIYMQ4Hnfq/wbBne3AiPIKBk8ezcelKQvbuj21zcMduBj3cmSuXLtOoQ1seea07o1/rD0DXIQOZ9fV4tq1exy3ZsmKjotKdyeFw8M7gwXR+/HHCwsKYPnMmixctImjPntg2D3fowOnTp2ncoAEtW7em91tv0bN7d3bv2sVDrVvjdDopULAgs+fNY8miRTid0bexeeLRRzl58mS6M15jHA469+/NR89250R4BO9OnsDfia7fLt6OuX6NO7Tj0dd68OVr/QC4cvky/ds+4bI8CbN1GdCbIc9050R4OIOnTODvpSs4kiBb/4efjM72aDs6vv4yX7zaN/Z4+5efZ+f6v92ST0TEG3lozkgR4HCc7WCgVsJGxpiXgFeBzMC9rjixV8wZKVWpPOGHgjkaHILzaiRr5y3krnvrx2uzc91fXLl0GYC9m7eSN6AgAIVvL4nDz59tq9cBcPnCxdh26VGlalUOHjjA4cOHuXr1KnNmzaJJ06bx2jRp2pQZ06YB8Pvcudxdpw4Aly5dii08brnlFqybu9tur1Qh3vVbM28B1RJcvx1xrl/Q5i2x18/dbq9cgfBDhzkafCQ629yFVLu3Qbw22+Nm2xQ/W4nyd5I7f162/LE2Q/KKiPyXGWO6GmM2xHl0jXs4iZck+gCz1n5prb0deBPo74pcXlGM5AkoyImw8Njtk2ER5ClYINn29du2YfPK1QAElijGhbNn6f75hwyaNpEOr/fAONL/tgICAwkNvT7UExYaSkBgYOI2ISEAOJ1Ozp09S548eYDoYmbewoXMmT+fAf36xRYnFhj/ww/8Ons2HTp2THdOgDwBBeJdvxOpXL8Gca4fQKbMmRk0ZQIDf/6Wao0bJPu6m5G3YAGOx80WHk6egOSzNWz3AJtW/glED3E9/uYr/DRshEsziYj4KmvtGGtt9TiPMXEOBwPF4mwXBUJS+HGTgAddkeumh2mMMWOstV1Tb5mWn5XU3qR7E+5u3YySFcsx5MnnAXD4+VOmWlUGtuvE8dBwXvzkfeo92JIV02elL1NSiRL0cJgkgl9rs+mff2jetCm3ly7N0E8+YfmyZVy5fJkObdsSERFB3nz5mPDDD+zbu5f169alL2tSOZJpe0/M9Xs/5voBvNK4DaeOHqNA0cL0+W4Uh3cHEXH4SLoyxQmXOFsyPUV1WjenVMVyDO7UDYAmHduzacUf8QotERFf4KFhmvXAHcaYksAR4FHgsbgNjDF3WGuvzVdoCezBBVIsRowxeZM7BLRI4XVdga4AX3/9daohToRFkDcwIHY7T2BBTkYcS9Su/N01aN21C0M6v0Dk1atAdC/KoR27OBocXbz9vXg5t1epCOksRsLCwihUqFDsdmChQkSEx/9QDAsNpVDhwoSFheHn50eOnDk5depUvDZ7g4K4ePEiZcqUYeuWLURERES/5+PHWTh/PpWrVk13MZLw+uUNLMipiKOJ2lW4uwZtuj7FB52fj71+AKeORl/ro8Eh7Fz3N8XLlXVZMXIiPIJ8cbMFBHAqiX/bCnfX5IFuT/Hek91is91RtTJlq1WlScf2ZMmWDf9M/ly6cJHJw0e6JJuIiFxnrY00xnQH5gN+wDhr7TZjzLvABmvtTKC7MaYJcBU4CXR2xblT6xk5ChwkfkeBjdlOdtJBTLfPta4f++dn36Z4kv1bdxBQvBj5ixTiZMRRajVvyle9B8Rrc1u5MnQZ+BafdHuFsyeuT/7ct3U72XLlImeeWzl78hTlalfnwNYdqbyt1G3etIniJUtStFgxwsPCaNm6Na++/HK8NosXLeKhdu3Y+PffNGvRgjV/Rg8vFC1WjNCQEJxOJ4WLFKFkqVIcCQ4ma9asOBwOzp8/T9asWalbvz4jP/883Vn3bd1OYPFiFChSmBMREdRufh+jEly/4uXK8NTAPgzr1pMzca5ftlw5uXLxEpFXr5Lj1tzc8b/KzB73fbozxWbbsp3A4rddz9aiKV++kTjbM+/04aOuL8fLFvc91H+wFSUrllMhIiI+wVPfTWOtnQvMTbDv7TjPe7rjvKkVI/uAxtbaQwkPGGMOJ9H+pkQ5nfzw/se8PnYEDoeDlTNmERK0n4e6d2X/th38s3QlHV7vwS3ZsvHSpx8A0UtQP+/+BjYqisnDRtB73EgwhoPbdrJs6q/pzuR0Ohn09tt8N3Eifn5+/DJlCnv27KHnq6+ydfNmFi9axJTJk/nk009ZvHw5p06d4pXu3eYnXWMAACAASURBVAGoXr063V58katXr2KtZWD//pw8eZJixYoxakx0jebv78/M335jxfLl6c4a5XQy8f1hvBFz/VbMmMWRoH20jbl+G5eu5NHXXyZLtqz0+DR6yfG1JbxFSpXgqXf6YKMsxmGYPXZivFU4rsg2/r2hvPnNCBwOP5ZPn8mRoH2069GN/Vt38PfSFTz2Rk+yZMtKz08/BOBYaBjDX3rNZRlERP5tfO0OrCallR4xy3dWWWs3JXGsh7X2izScw3Ypn2hlkFcYv30tpYsX93SMZAUdPEin8jU9HSNJ32+PHlp6vFwNDydJ2o871ns6goj8tyQ5u9Fdnqtwt1uqkbHbVmfo+0ir1HpG1gGxEyWMMU8C7YgeunnHfbFERER8lyfuwOpJqa2B/Rq4AmCMqQ98CEwETnN9ToiIiIjITUutZ8TPWnsi5nkHYIy1dhowzRjzj3ujiYiI+CZfmzOSWs+InzHmWsHSGFgS55hX3EpeRERE/t1SKyh+BpYbY44BF4GVAMaY0kQP1YiIiIiL+VrPSIrFiLX2fWPMYqAQsMBeX3rjAHq4O5yIiIj896U61GKtXZPEvt3uiSMiIiK+RvM+REREvIyW9oqIiIhkIPWMiIiIeBlNYBURERGP8tQX5XmKhmlERETEo9QzIiIi4mWifGyYRj0jIiIi4lHqGREREfEyvjZnxFj3v2HfuqIiIvJfZDLyZG3KVHXLZ+fM3f9k6PtIqwzpGWl35/8y4jQ3bNrOv6lb6k5Px0jWqn076VK+lqdjJGn89rUANL+jkoeTJG3eni1c/m2Ap2Mk6ZYHBns6goiIV9EwjYiIiJfxtfuMaAKriIiIeJSKEREREfEoFSMiIiLiUZozIiIi4mWibJSnI2QoFSMiIiJeRndgFREREclA6hkRERHxMr52B1b1jIiIiIhHqWdERETEy/janBEVIyIiIl4mSsM0IiIiIhlHPSMiIiJexrfuMqKeEREREfEwrylGqta9hxHzpjNy/m889FyXRMdbd3mcz2ZPZfhvkxn43VcUKFwIgBJ3luGDSeP5bNYvDP9tMvc0v88t+WrVr8tPi+Yxacl8nnj+uUTHq9Sozrczp7Fs91YaNr8/0fFsObIz48/l9HrHPV9rX6lubYbMmcJHv0+l5bNPJjp+f+eOvD9rEoNn/EDvcSPJVzgw9ljeQgG8PnYEH8yaxPuzJpE/5tq6SrV6dRg7fybfLprDw12fSXS8Yo1qfPHrZGbv2EjdZk3jHXv6jV6MnjOd0XOmU79F4uvqaqt2hdJ66DxafjSXb5fuSHQ85OR5nh2zjHbD5/P0V0sJO3XB7ZlExPdEWeuWh7fyimEah8PBc2+/ybtPv8jx8HA++uUH1i9ZTvDe/bFt9u/YRe/2T3Dl0iXuf7Q9nV7vyfBX3+LypUt88eYAQg8eJk/B/Ayb+iP/rPqTC2fPuTTfq4PepteTTxMRFs43v/7CqkVLOBC0N7ZNeEgoH/TuQ8dnn07yZzzXqyf/rFvvskxxGYeDTv3fYNizPTgRHsHAyePZuHQlIXGu38Eduxn0cGeuXLpMow5teeS17ox+rT8AXYcMZNbX49m2eh23ZMuKjXJdB6HD4eCld/rRt0tXjoWF8fm0SaxdspRDQfti20SEhPLJmwNo90zneK+t0bAet1cox0ttHiZT5swM/fE7NqxYxYVz512WLy5nVBQfzPibMc81ICB3Vjp+sYiG5Qtze0Du2DafzN5E6/+V4IHqJVgbFM6I37fwwaO13JJHRMRXeEXPSOnKFQk7FEx48BEir0ayau58ajRuGK/N1rUbuHLpEgC7N20hX2BBAEIPHCL04GEATkYc4/SJk+TOm8el+cpVqUzwwUOEHA4m8upVFs2eS92mjeO1CTtyhL07dxMVlbjyLFuxAnny52Pdyj9cmuuaUpXKE34omKPBITivRrJ23kLuurd+vDY71/3FlUuXAdi7eSt5A6KvX+HbS+Lw82fb6nUAXL5wMbadK5SpXImQg4cIOxxM5NVIls+ZR+3GjeK1iTgSwoFdu7EJqvbbSt/OlnUbiHI6uXzxIvt37qJavbouy5bQ1sMnuC1/Dormy0Emfz+aVbmNpdtC4rXZF3GGWqWjr13N2wuydNsRt+UREfEVKRYjxhg/Y0w3Y8xgY0ydBMf6uypE3oACHAsNi90+ERZBvpgPy6Q0bv8gf69I/MFeulIF/DNlIuxQsKuiAVAgMICI0NDY7aOhYRQICEjTa40xdO/7JqM+HObSTHHlCSjIibDw2O2TYRHkKVgg2fb127Zh88rVAASWKMaFs2fp/vmHDJo2kQ6v98A4XFej5g8syNE4/7bHwsLJl8Zrt3/nLqrXr8stWbKQK8+tVK5dkwKF0vbamxF++iIBubPFbgfkzkrEmYvx2pQpdCuLtkb/fi3eeoTzlyM5dd51xZuICETfZ8QdD2+V2qfO10AD4DgwwhgzPM6xtsm9yBjT1RizwRizYcyYMamGMJhE+xL+lXxN/dYtuL1CeX77dmK8/bcWyM/LQwczsu87yb72ZiVOl3y+hB564jFWL1tORJwPZFczSQVM5pfu7tbNKFmxHPPG/QCAw8+fMtWqMnnYCAY98hQFihah3oMtXZkuiWhpu3Z/r1rNhuUr+WTK97z56VB2btyE0+l0YbbUJUz/Wssq/LXvKI98toAN+45SMHdW/BxJ/gOIiEgapTZnpKa1tjKAMWYkMMoYMx3oSNKf0QBYa8cA16oQO3/4Vyme5Hh4BPkLxZlQGViQExFHE7WrfHdN2j3/DAM6PUvk1aux+7Nmz06/rz7n589GsWfTllTe0o2LCAunYKHrkzoLFArkWEREml5b8X9VqVKjGg898RhZs2UjU6ZMXLxwnq+GDk/9xWl0IiyCvIHXewzyBBbkZMSxRO3K312D1l27MKTzC7HX72RYBId27OJocPRwxN+Ll3N7lYowfZZLsh0LC6dAnH/b/IEBHE/jtQOYNHosk0aPBaD38I8IOXDIJbmSEpA7K+Gnr09IDT99kQK5ssZrUzB3Vj59MrqT8MLlqyzaGkzOrJndlklEfJM3TzZ1h9R6RmL/X9ZaG2mt7QpsApYAOVwVImjLNgoVL0bBIoXxz+RP3Rb3s2HJ8nhtSpYrS7dB/fjwxVc4c+Jk7H7/TP70HvkJy36bw+r5i1wVKZ6dm7dQrERxChUtgn+mTDRp1YI/Fi1J02vf7fUG7erey8P1G/PlkKH8PuM3lxYiAPu37iCgeDHyFymEXyZ/ajVvysalK+K1ua1cGboMfIvPu7/B2TjXb9/W7WTLlYuceW4FoFzt6vEmvqbX7i1bKVyiOAFFi+CfyZ8GLZuzZvGyNL3W4XCQ89boyaMlypahZNk7+GvVny7LllCFonk5eOwcwSfOcTXSye+bDtGwfOF4bU6evxw7L+ibpTt5qHpJt+UREfEVqfWMbDDGNLPW/n5th7V2kDHmCDDaVSGinE6+GfwRA779EofDwZJpMzkctI9HezxP0NbtbFi6giffeIUs2bLx2mdDATgWGsaHL/binmb3Ub76XeS8NTeNHmoNwMg+Azmwc7er4uF0Ohn+zmCGT/gWh8PBnF+msX9PEM+80oOdW7byx+Kl3Fm5Ih+MHknO3Lmo07gRz/TsTqdmrV2WISVRTic/vP8xr48dgcPhYOWMWYQE7eeh7l3Zv20H/yxdSYfXe3BLtmy89OkHABwPCePz7m9go6KYPGwEvceNBGM4uG0ny6b+6tJsowd9wHvjvsLPz48FU2dwKGgvnXq+xO4t21i7ZBllKlVgwKjPyZErJ7UaNeCJl1/k+RYP4efvz8c/TwDgwrlzDHu9D1FuHKbx93PQ94H/8cI3K3BGWR6sUZLSgbn5cv5WyhfNQ6MKRVi/N4IR87ZgDPyvZAH6PfQ/t+UREd/lzfM73MGkNPfBGFMDCLbWhsZsPwm0Aw4C71hrT6ThHLbdnd75f9jTdv5N3VJ3ejpGslbt20mX8t65bHT89rUANL+jkoeTJG3eni1c/s0993RJr1seGOzpCCJy4zJ0clj1kqXdUo1s2B/klZPc0jKB9TKAMaY+8CEwETjN9TkhIiIiIjcttWEavzi9Hx2AMdbaacA0Y8w/7o0mIiLim5K4ZdV/Wmo9I37GmGsFS2OiJ65e4xV3bxUREZF/t9QKip+B5caYY8BFYCWAMaY00UM1IiIi4mK+NoE1xWLEWvu+MWYxUAhYYK/PdnUAPdwdTkRExBepGEnAWrsmiX2uWzcrIiIiPs0rvihPREREfJeKEREREfEorYgRERHxMj721TQqRkRERLyNr01g1TCNiIiIeJR6RkRERLyMb/WLqGdEREREPEw9IyIiIl7G1+aMGOv+Kbu+dUVFROS/yGTkye4sXsItn507Dx7I0PeRVuoZERER8TK+9ld8hhQjpUqWy4jT3LB9+3fQovV3no6RrLmznqJ5i9GejpGkeXNfAKBSxcYeTpK0LVsXU7Z4CU/HSNKugwcAuKtKS88GScbGTXM8HUHE5/laMaIJrCIiIuJRGqYRERHxMr42gVU9IyIiIuJRKkZERETEozRMIyIi4mV8a5BGPSMiIiLiYeoZERER8TLqGRERERGfZIxpZozZZYwJMsa8lcTxW4wxk2OOrzXGlHDFeVWMiIiIeBnrpkdKjDF+wJdAc6A80NEYUz5Bs2eAk9ba0sCnwEfpeZ/XqBgRERHxMp4oRoCaQJC1dp+19gowCXggQZsHgAkxz6cCjY0x6f6+GxUjIiIiAlAEOBxnOzhmX5JtrLWRwGkgX3pPrGJERETERxhjuhpjNsR5dI17OImXJOxQSUubG6bVNCIiIj7CWjsGGJPM4WCgWJztokBIMm2CjTH+QG7gRHpzebwYeXtgXxo2rM+lS5d44/W+bNu2PVGbihXLM2zYEG7JcgvLlq3g3UEfADDii+GUKlUCgFy5cnHmzBlatWxLkSKFWbhoDvv27Qfgn42b6N9/ULqzdutaixrVinL5ciTDP1/F3r3HE7V5952m5M2bDT8/w7Zt4Yz6ag1RUZZSJfPS/cW7yZTZjyin5cvRq9m951i6M8X1fLc61KhRnMuXI/lk+BL27k388we/2zImn4Ot20IZNWolUVGWt95qStEitwKQI0dmzp27Qvcev7gs21t9XqJevVpcunSZ/v2GsmPHnkRterz8NG3aNCVXrpzUqtkqdn9gYEHe/+BNcubMjp+fH599OpaVK9fddJZ6DRrQb+DbOPz8+GXSZMaOjv/NyJkyZ2bo8OFUqFSRUydP0at7d44EB1OkaFHmLl7E/r37ANi0cSMD+/UjS5YsfD56FLfdVhxnlJOlixbzyUcumdOVSO83u1GnbnUuXbrMwAGfsnPn3njHs2S5haHD+lC0WCBRUVGsWL6OEZ+Pd0sWEXGndE/DuBnrgTuMMSWBI8CjwGMJ2swEOgOrgfbAEmvtv7tnpGHD+pQoUZx7GzWjatUqDH7vbdo+9GiidoPfG0jfvgPZuPEfxn33NQ0a1GP58pW83OPV2DZ9+/Xm7JlzsdsHDx6mVcu2LstavVpRihTOxbPdplG2bAG6v3A3vV6fnajdkI+WcfHiVQD69WlE3TolWLFyP08/VZ2fJv3Dhr+OUL1aUZ5+qjpv9f3dZflqVL+NwkVu5Zlnf+LOsgF0716fXr2mJ843ZAEXruXrdz/16t7O8hVBfPjhwtg2zz57NxfOX3FZtnr1alL8tqK0bPEklSuXo/+Anjz+WPdE7ZYvW83PP/3KnLkT4+3v1u1x5s9fxpTJsyhVqjijRn9As/sfv6ksDoeDtwe/y1OPP0F4WBhTZ85kyaKF7N0TFNvm4Q6PcOb0ae5r0JAWrVvz+ltv0at7dN5DBw/yYIsWiX7uuDFjWbt6NZkyZWL8Tz9Sv2FDVixbdlMZk1O3bnVuu60wD7R+jkqVytK3/0s8+cSridpNnDidDes34+/vz9dj36dOnWr88cdfLs0iIv891tpIY0x3YD7gB4yz1m4zxrwLbLDWzgS+Bb43xgQR3SOS+EP7Jnh0zkiTpvcyY/pvAPzzzyZy5cpFgQIF4rUpUKAAOXLkYOPGfwCYMf03mt7XONHPatGiGbNmzXFb1tq1b2PxkugPrF27jpI9e2by5MmaqN21QsTPz+Dv74gdSbMWsmXNDED27Jk4ceKCi/OVYPHiXQDs3BVOjuy3kCdPtkTtLsTmc5DJ34FNYqivfr3SLFselGj/zWrUqA4zZy4AYPPmHeTMmYP8+fMmard58w6OHUvc22ct5MieHYCcObNz9GjiHqm0qly1KgcPHCT48GGuXr3KnFmzaNz0vnht7m16HzOmTQNg/ty53F3nnhR/5qVLl1i7ejUAV69eZfvWbQQEBt50xuQ0aFSb2bOWALBlyy5y5sxO/vx5EmS5zIb1mwGIjIxk5469FAzI7/IsIuJuxk2PlFlr51pry1hrb7fWvh+z7+2YQgRr7SVr7cPW2tLW2prW2n2ueLceLUYCAwIIDQ2L3Q4LDSMwsGD8NoEFCQsNv94mLJzAgIB4bWrUrM7xY8c5cOBg7L5ixYowa/Y0fp40kRo1qqU7a/582Th67Hzs9rHj58mfL/GHPcDgQffx0w8duXjxKqv+PADAmLFrefrp6kwY9wjPPF2D8RNc+5dqvvzZOXb0es/QsWPnyJ8/e5Jt3xvckp9/6sKFi1dZtSr+71HFioU4eeoCISGnXZatYEB+wsKOxm6Hhx+9oQ/IUaMm0KpVYxYtmsSoUR8w5IMvbjpLQGAAYaHXh0DDQ0MJCAxI1CY0JLqN0+nk7Nmz5MkT/aFftFgxZsydw/eTJ1OtRo1EPz9nrlw0atKY1X/8cdMZk1OwYD7CwuNex2MULJj8JPYcObNTv0Et1q3d5PIsIiKulGIxYozJZozpbYx5wxiTxRjTxRgz0xgz1BiTI70nT2ppcsKhpyTbJPhrvk3rlsyM0yty9OhR6tZpTOtW7Xj/vQ/59LNh5MiR9AdzeiQ3SjZg4AKeeHIymTL5UaVyIQBatLiTsd+so/PTUxj7zTp6vlzXpVlMEhVvcsN4/QfM4fEnJkTnqxJ/1VbDBnewfJnrekUAklyBfgNDjC1a3Muvvy2gSZNHefHFvnwwpE+SvxdpypKG65Tc72VERASN7r6Hh1q05MPBg/lkxOdkz3H9PwM/Pz+GfzGC778bT/Dhw4l+RnolnT3ptn5+Dj78sDc//zSTI0fCkm4kIuIlUpszMp7o9cRZgTnADuBjoDUwGuiU1Itilgp1Bfj666/jHevU6TE6PNoegM2bt1Ko0PXu7MBCgYTH+csPIDQ0nMBC1/9yDQwMIDw8Inbbz8+P+5s1oU3r9rH7rly5ypUrpwDYunU7hw4dpmTJEmzZsi2VtxtfqxZ3cv/9ZQDYs+cYBeL0NOTPl53jKQy1XL3qZM3aw9SudRsb/wmhyb2l+XrMWgBWrjpAzx51bihLkvlaVaDZ/dE3x9u9J4L8Ba5/MObPn4Pjx1POt3bNAWrXLsHGjcEAOByGe+4pycsvT013tkcffYB27aPnVmzduovAwOvDbwEBBYiISPtQy0Ntm/P889F3Jd60aTu3ZM5Enjy5OXHi1A3nCgsLI7BQ4etZChUiIs7vE0T30BUqXJjwsDD8/PzImTMnp05Fn+vUlei5NNu2buXQwUOULFmSrVu2ADD4wyEc2L+fCePG3XCu5DzSoSVt2zaLPue23QQGxL2O+ZMdsur/dg8OHQrhpx9/c1kWEclIHpnA6jGpDdOUsda+BrwEVAB6WGtXAL2BKsm9yFo7xlpb3VpbvWvXrvGOff/9T7Rq2ZZWLduycMFiHmobfXO3qlWrcPbsWY4ejV+MHD16lPPnzlO1avTpHmr7AIsWLok9XqfO3ezdu5+wsOtDOXnz5sHhiH5rxYoVpUSJ4hw6FJzKW01s9tyd9Og5kx49Z7J6zSEa31sagLJlC3D+whVOnrwYr32WLP6x80gcDkON6kU5HBw93HH8xAUqVYwuvKpULsSRkDM3nCdRvtnb6N7jF7r3+IXVq/fTuHFZAO4sG8D585c5eTJ+MRKdL1tsvuo1biP48PUP9LvuKkpw8CmOHT9Pek2a9BsPt+/Gw+27sWTJH7RpEz0vo3Llcpw7dz7JuSHJCQuNoHat/wFQstRtZL4l800VIgBbNm2iRMkSFC1WlEyZMtGydWuWLFwYr82SRQt5qF07AO5v0YI1f/4JQJ68eWN/r4oWK0aJkiU4fOgQAK+8/ho5cubkg0Hv3lSu5EyZPIdHO/Tg0Q49WLp0Da1a3wtApUplY67jyUSvefGlTuTMkZ1hQ5NbvSci4l3StJrGWmuNMXOvLd+J2U73Up6lS5fTsFF9li6bz6WLl+jdu2/ssdlzpseuhhkwYBBDhw0hS5ZbWL58JcuWrYht16p1C2bNjD9xtWbN6rzS62Wczkiczij693+H06fTNwdi/YZgalQvyrdj2nH5spNPP18Ze+yLz9vQo+dMsmTxZ+CAxmTy98PhZ9i0KZS583YCMGLkH3R7rhZ+fg6uXnHyxcg/05UnUb71h6hRozjjvn2MS5cj+fTTpbHHRn7xMN17/EKWLJl4Z2BzMmXyw+EwbNp0hDlzr/cWNahfmmXLEy+5Ta+VK9ZSv14t5s77nksXL9F/wLDYY79M/ZqH23cDoNerXWnZ4l6yZLmFRYsmMW36XEaPmsiwYV/xzqBX6fRkO6y19O8/9KazOJ1O3n37bb6ZOBE/Pz+mTZlC0J49vPxqL7Zu3sKSRYuYOnkKwz4dzoLlyzh96hS9uvcAoEatmrz86qs4I504o5wM7NuP06dPExAYyAs9erA3KIgZc6J/F3+YOIGpkyan46oltmrleurWrc7M2d9w6dJl3nn709hjkyZ/waMdelCwYD6e6/oo+/Yd5udJIwCYPGkWM2YscGkWEXEz3+oYwaS0PNgY8w3wirX2XIL9twMTrLVpmfhgS5Usl76UbrJv/w5atP7O0zGSNXfWUzRvMTr1hh4wb+4LAFSqmHhlkzfYsnUxZYuX8HSMJO06eACAu6q09GyQZGzc5L5VaSL/YhlaHpQoUTrdf/An5cCBIK8sc1LrGfkayAGcAzDGPAm0Aw4BD7o3moiIiPiC1OaMfA1cATDG1Ac+BCYCp4Cv3BtNRETENxk3/c9bpdYz4metvTbTsAMwxlo7DZhmjPnHvdFERETEF6TWM+IX80U4AI2BJXGOefx7bURERP6TjHHPw0ulVlD8DCw3xhwDLgIrAYwxpQHX3aJTREREYnnzkIo7pFiMWGvfN8YsBgoBC+J8M58D6OHucCIiIvLfl+pQi7V2TRL7drsnjoiIiHj4q+MynG+9WxEREfE6KkZERETEo1SMiIiIiEdpea6IiIiXMV68DNcdVIyIiIh4G+NbAxe+9W5FRETE66hnRERExMsYH+sr8K13KyIiIl7HXL+pqtu4/QQiIiJulqEzSu8oXcUtn517gjZ55czYDBmmaVmmSkac5obN2b2JPpUaejpGsoZsWcboKk09HSNJL2xaCECn8jU9nCRp329fx5/16nk6RpLuWbkSgEOdqns4SdJu+34DT07529MxkjTxkf95OoJIxtAEVhEREZGMowmsIiIiXsaoZ0REREQk46gYEREREY/SMI2IiIiX0X1GRERERDKQekZERES8jCawioiIiGQg9YyIiIh4GWP8PB0hQ6lnRERERDxKPSMiIiJextfmjKgYERER8TK+Voz41rsVERERr6OeERERES/jaxNYvbYYqVbvHrr2exOHn4MFv8zglzHj4h1/8KlO3P/wQzgjnZw+eZLP+gzkaEio2/KUqVOTVm92x+Hnx/rpc1j+7U/xjtd8uA13d3yQKGcUVy5cZMagj4nYd5CiFe/koYGvA2AMLBo1nu1LVrk8X7F7qlP3zRcxDgc7Zsxj47jJSbYr1aQe93/yNlM7vsTR7bu5JXdO7v/kbQpWKMvOmQtYNWSky7NVqlubTn1ew+HnYNnU35j9zcR4x5t1foyG7dvgjHRy9uQpxvYfzPGQMADyFQrgmXf7kTcwALB83K0Xx1z873xrzZqU7NkTHA4iZs/myI8/xjteoHlzSrz4IleOHgUgdPp0ImbPjj3uly0bVX/4gRMrVrD/s89cmi1LpbvJ0+l1cDg4v+xXzsyekKhNtppNyN22K9Zarh7aw/HR/aNz5Qsg7zMD8M8bfe0iPu6J85jr/xt54q6iVAnMxWWnZey6Axw8dTH+e/B30K9RmdjtvNky8+fBE/z4T3DsvhpFb6XHPaUYuHAn+09ecHlGEfFuXlmMOBwOXhjYl/5PdeNYWDifTvuJNYuXcXjvvtg2+7bv5JW2j3H50iVadHyYp3v34qNXerslj3E4aNOvJ992fZ0zYUd5adJX7Fj6BxH7Dsa22TR3Eet+mQlAuYb30PKNl/juhd6EB+3ny0e7EeV0kjN/Xl6e+i07l68myul0ab56fXswq9ubnA8/RrufRnJg2WpO7jsUr12mbFmp9NiDhG/eEbvPeeUq674cT97SJclbuoTLMsXN1rl/bz56tjsnwiN4d/IE/l66kpC9+2PbHNyxi7cf7syVS5dp3KEdj77Wgy9f6wdAtyHvMPPr79i6et3/2bvv+Kaq/4/jr3OTTmhLSyeFsil7CEjZZSmggqgITlDZQ0BRQBQRRVQ2iggCrq+KyBKZslrKBmUW2rK7d+mgM8n9/dGSNjSlICmtP87z8eBB7r2fm7yT3NucnHvuDTb2dqgGg2UDKgp13nqL4EmTyE1IoPm335J88CBZ166ZlCXu2VNiQ6PGsGGknTpl2VwAQsF5yBTiPx+LPjkOz1k/kvnPfnTRha+d1qMGjk+9RuysN1Az01EcnY3Lqo6cRdrm1WSfO4qwsQPVwq8d0NzTEY/KNryz/Tx1XewZ2tqHj/aEmtRk6wx8sCvEOP1Rz4actI+w1QAAIABJREFUiLxhnLbVKvSq78alpJsWzydJ/1VyzEgF0KB5U6KvRxAbEYUuT8f+rTvw6+lvUnPm6HFysrMBCDl1FlcP9zLLU6NZQ5LCo0iJjEGv03F6+14adetoUpNzs/DbnLWdLSoqAHnZOcaGh9bG2jjfktyb+pIaEU16VCwGnY5LOwKo5d+hWN2jY4dy6vu16HJyjfN0WdnEngxGX2SeJdVt1oS48EgSIqPR5+k4sv0vWnfvYlJz4djf5GbnAHDpzFlcCt7LanVro2g0nDt8DICczCxjnaVUbtSIrKgocmJiUHU6EvfswaVTp7tev1KDBli7uHDj+HGL5gKwrtsEXVwE+oQo0OvIPPIX9q27mtRU7jaA9N1rUTPTATCkpQCgrVYbFA3Z544CoOZkoeZa9rUDeMTbiYPXkgG4nJyJvZUGJ9uSv+N4VLbB0VZLaGKGcd6zTauxLSSOPL3lG0uSJP033HNjRAgRVhZBiqrq4U5ibKxxOjE2nqoeHiXWPzZwACf2HyyzPI7ubqTGJhin0+IScPJwK1bnN/hpJm/7md5vjeLPOUuM82s0a8TEjd8xYcN3bJq1wKK9IgCV3F25WSTfzfhEKnm4mtS4NqxLZU83ru8/atHHLo2zhxvJsXHG6eTYeJzdi792t3R9ph9ngg4D4FXLh8z0DN5c/Dkfr/+JwZPHIxTLtp9t3NzIjY83TucmJGDt6lqsrqq/Py2+/x7fjz/G2r2g4SsEtcaN49rXX1s00y0aZ3f0yYWvnS45Ho2zaaNb6+mDlVdNPD5YhceH32HbrD0AVl4+qJnpuL75BZ4f/0yVwW9CGXzTcrGzJjmrsCGbnJWLi511ifV+Ps4cjUgxTtesYoeLvRWnYtIsnk2SpP+OO/51EkKkCyHSCv6lCyHSgbq35t9hvRFCiBNCiBMrVqy451BCiOIzVfM9Ct36PUH9po1Zv/L7e36cuw9kLk7xPEfWbGJe35fYsXA53Ue8YpwfcfYCiwa8xtLBI/Ef9hJa65L/WP+7fKW8XkLQYfJoDs1fbtnHvQvm3suS+oY6PNWb2k0bsXX1TwAoGg2+rVvy69zFfPj8UNyre9Pl6SfLMK15KQcP8vfAgZweOpQbJ05Q/733APAcMICUI0dMGjMWZeZtvX0/EIoGrUcN4j4dQeLX03EZ9j7CvjIoWmx8W5Hy62JiP3wVrXt1KnV5qmxy3h7xDr1/fjWcORKe35MigBdbVufXU1EPJJck/ZcIoSmTfxVVaWNGvgecgHdUVY0DEEJcVVW19p1WUlV1BXCrFaL+MW/pPYVKjI3D1dPTOO3q6U6SmT/4LTu0Y9DoYUx56Q10eXn39Bj3Ii0uASfPwm/zjh5upMUnllh/Zvtenn5/UrH5CVfDyc3KxqNebaLOh5pZ89+5GZdApSL5Krm7cjM+yThtXckOl3q16LdyHgD2ri70WTyL7RNmkHC+bDu6kmPjCwaf5nPxdOdGfEKxuibt29JvxGt8OmSU8b1Mjo3n+oVQEiKjAfh7TyD1WjQlcIPl8uUkJBT2dADWbm7kJpq+t7q0wnZ33J9/UnPUKAAcmjTBsUULPJ9+Go2dHcLKCn1WFuHLLdPo0yfHo3EpfO20Lu7ob5i+drrkeHIvnwW9Hn1CNLqY61h5+KBPjiP3emj+IR4g8+8AbOo15Wbg/efqUc8V/9r5vUdXUzILekLyx3u42FmTkmV+X6zhZIdGEVxLyR/gamulUN3Jjmnd6gPgZGvFxE51WHTgihzEKkkPmTs2RlRVHS+EaA38KoTYBHxFyV9sLSbsbDDetXzwqO5NUlwcXZ7ozdy3ppnU1GnUkHGzPmDGG2NITU4u0zyR50JxrVkdZ29P0uISadGnO2umfGJSU9XHm6Tw/D/8vl38SCy47eztSWpsAga9nipeHrjVqkFKdGyxx7gf8cGhVPHxxsHbk5txidTr7c/uaXOMy3MzMvne/znjdL+V8zi8YEWZN0QArpw7j2fNGrh5VyM5Ph6/Po/x9bsfmNTUbNSA1z6cxtyRE0hLTjFZt5KjIw7OVUhPuUFjvzZcPXfh9oe4LxkhIdhVr46Nlxe5CQm49uhB2EcfmdRYVa1KXlJ+486lY0eyrucPXL748cfGGrc+fajs62uxhghA7pXzWHnWQONWDX1yPPZ+j5H09fsmNVl/B2Df/nFuBm1BqeyE1tMHXUIUhpvpKJUcUByqYEi/gW3jNuRetcxrt+dSInsu5TfYWng50rOeG0ciUqjrYk9mnp7UbJ3Z9dr7OHM4vPD9zcozMPaPM8bpaf71WXM6SjZEJAl5am8xqqr+LYToCYwDAgHbsg5l0OtZNmsOH69ahqJR2LVuE+GXLvPym2O4eC6Yo3sDeWPKJGzt7Zm2ZC4ACdGxzBo9oczybP50Ma9/MxehUTixcTvxl6/Rc+xrRAWHciHgEO1fGEA9v9bodXqy0tL5fXp+Y6BWq2Z0feNF9Do9qsHAH7MXkXkj1aL5VL2BoDlf8eSyOQhFIWTTTlIuX6ftmCEkBIdxLfDwHdd/adtPWFe2R2NlRe1uHdgyamqxM3H+LYNez4+z5/LOt0tQFIX9G/8k6tIVnhk3gqvBFzi5L4jBk9/E1t6O8QvzX7Ok6FgWjpuMajDw69zFTF29FCEE14JD2Lduk0VyGen1XFm4kMbz5yMUhbitW8m6do0ab7xBRkgIKQcP4vXcc7h07Iiq16NLS+PSp59aNkNJDHqSf5yL+ztfgqLh5v7N5EVdwemZkeRevUDWyf1knz2MbTM/vD5bi2owcGPNEgwZ+dvXjV8X4z51GQhB7rULZOzbaPGIp2PSaOHlxNy+TcjVGVh5vPAMs497NTQ5i+bRGs7MD7pk8QyS9P+R8pCdTSPMjX0wLhSiLRChqmpswfQ04CNgGfCRqqp30yWhPtGghSWyWtzWsNNMa+Zf3jFKNOdsAMta9CrvGGaNPr0LgFcaP1rOScz76fwxDnXuXN4xzOoQFARA+CttyjmJeT4/neDVtf+Udwyzfnz+kfKOID28zI3iKjOtWjxRJkchTp7e+kCfx90qrem1HMgFEEJ0AcYDLwBpFI4JkSRJkiTJguQAVlOaIr0fg4AVqqquB9YLIcrgKk+SJEmSJD1sSm2MCCG0qqrqgB7AiHtYV5IkSZKkf6Ei92KUhdIaFL8CgUKIRCALCAIQQtQDLDsKU5IkSZIkQDZGTKiqOlsIsQfwAv5SC0e7KuSPH5EkSZIkSbovd3Nq7xEz88r+AhWSJEmS9JASysPVM/JwncgsSZIkSVKFIxsjkiRJkiSVK9kYkSRJkiSpXMnTcyVJkiSpglHk2TSSJEmSJJWnh+3UXnmYRpIkSZKkciV7RiRJkiSpgpE9I5IkSZIkSQ+QKLyoapkp8weQJEmSpDImHuSDdfAbViafnYeOrHygz+NuycM0kiRJklTByLNpysAbjf0exMPcs1Xnj9CpTsPyjlGiA1dC+KVlr/KOYdaLp3YBsLhFxcw34fQu9vl1Le8YZnU7EgjA2826lHMS8+af3U/ffj+Wdwyztm1+FYCwAW3LOYl5DTYeL+8IkvSfJHtGJEmSJKmCkb9NI0mSJEmS9ADJnhFJkiRJqmCEeLg+nmXPiCRJkiRJpRJCuAghdgkhLhb872ympqYQ4m8hxCkhRLAQYtTd3LdsjEiSJEmSdDemAntUVa0P7CmYvl0M0EFV1ZZAO2CqEKJaaXf8cPUDSZIkSdJ/QAW9Amt/wL/g9g9AADClaIGqqrlFJm24y04P2TMiSZIkSdLd8FBVNQag4H93c0VCiBpCiDNABPC5qqrRpd2x7BmRJEmSpAqmrAawCiFGACOKzFqhquqKIst3A55mVp1+t4+hqmoE0Lzg8MwmIcQ6VVXj7rSObIxIkiRJ0kOioOGx4g7Le5a0TAgRJ4TwUlU1RgjhBcSX8ljRQohgoDOw7k618jCNJEmSJFUwitCUyb/7tBkYUnB7CPDH7QVCiOpCCLuC285ARyC0tDuWPSOSJEmSVMEIpUJ+PH8GrBVCvAGEAwMBhBBtgFGqqg4DGgHzhRAq+T8uOE9V1bOl3XGFfLaSJEmSJFUsqqomAT3MzD8BDCu4vQtofq/3LRsjkiRJklTBPGxXYK0wz7ZpJz9emDYJoVEIWreZ7St/Mln+2JAX6PxcP/Q6PRkpKXz3/mySomPxffQRBk+daKzzql2T5ZM/4OSe/RbN165LJybMmI6iKGxZu47/ffOtyfIWbdvw5gfTqNvQl5kT3iZg+07jssCLwVwJDQMgLjqGqSPGWDQbgFeHNrR+dwxCUbi8cTvnv/vNbF2Nnp3pPG8GO14cS/L5MCpV8+CJDatIvx4JQOKZCxyfvdii2Wp2aEPXKfnZgjdu58Rq02zNBj5J80H9UPUG8rKy2DNrIclXwlG0WnrMmIh74waoBgOBX3xN1IkzFs0G4OL3KPUnjQdFIWbzVsJ/+sVkuecTvak7bjQ5CQkARK3bSMzmrVSuX48G776FtpI9qsHA9e9/In73Potm8+34KE9PeRNFo3B0w1b2rvrZZHn7gf3o+MIzGPR6cjOz+P2jucRduU6D9m3oO3EkWisrdHl5bJm/jEvH/rFoNnNGDm9L2zbe5OToWbDoIJevJJdYO2N6Nzw9KzNm/J9lksW+VXvc33gbFIXU3X+QsuGHYjWVO/Sk6uDhoELOtTBiF36ATa0GuI+agmJXGQx6ktZ9R8bBXWWSUZKkfBWiMSIUhZfen8z8YW+SEhfPB799x6l9QcRcvmasuX4hlICBQ8nNzsF/0DM89/Y4lr/9PqHH/uGjZ/J/VrySkyNzdvxO8MGjFs2nKApvfTSDSa++TnxsHCs3/c6B3Xu5dumysSYuOoZP353GC8NeL7Z+TnY2rz05wKKZihKKQptp49k7agpZcYk8/vNXRAYeJu1KuEmd1t4O3xeeJvHMBZP5GZHRbB90V1fs/VfZ/N8bz8aRU8iIS2TwL19xJeAwyUWyhW7by9nftwBQu2t7Ok8exR9j3qPps30B+Pm5Edi5VKH/0tmseXEcqKrlAioKDSZP5NSbb5MTn0Cb75aTGHSQzGvXTcrid+/l4nzTRpo+O5sLs2aTFRGFtWtV2nz/LclHjqPLyLBINKEoPDN9EstHvEVqbAIT16wgeN8B4q4UZvtn224O/74ZgCb+Hen3zji+Hf0ON1NSWT1uKmkJSXjWq82Ib+Yxq+ezFslVkjatvfGu5siwkZvw9XVl3Oh2THpnu9naDu19yM7WlV0YRcF9xLtEzRxHXlIcNb/4gZvH9pMbedVYYuVVA5dnhxIxbRiGm+lonPKvbG3IzSZ28UzyYiLQOLtSc95PZJ48jCHTMu+rJN2NCnrRszJTIc6mqdOsMfHhkSRGRqPP03Fs+y5ade9iUhN67B9ys3MAuHLmHM4exa+10vqxbpwNOmKss5RGLZoTeT2c6IhIdHl57N6yjU69TA+bxUZFcTkkDIPBgh+Ud6lqU18yIqK5GRWLQafj+s4Aqvt3KFbXfOxQzn+/Fn1urpl7KRseTX1JjYgmrSBb2I4A6tyWLfdmpvG2lZ2tsbHhUqcm4UdPApCVfIPc9Jt4NGlg0XyOjRuRFRlFdnQMqk5H3K69uHbpdFfrZkVEkhURlf8cEpPIS0nBytnJYtl8mjUiKTyK5MgY9DodJ7fvoUk302w5RV47aztbVPJfu6iQi6QlJAEQe+kqWhtrNFZWFstmjl+7GuzZl99ADw1NpFIla5yd7YrV2dpqGdC/Mb+utXwvl/Ex6jchLyaCvLgo0OlIO7CLSo92Nalx6vU0N7b/juFmOgD61BQA8qLDyYuJyJ+Xkog+NdnYUJGkB0UIbZn8q6gqRGOkiocbybGFpyunxMZTxd2txPpOzzzFuaDDxeY/2qcXR7f+ZfF8bp4exMfEGKcTYmJx8/C46/WtbWxY+cc6lq9fQ+dexcb+3Dc7d1duxiYYpzPjErF3dzWpcfati72HG9FBxXuNKnt70nvNMnqsnI9bq6YWzVbZ3ZX0Itky4hOp7OFarK75oH4M2fIDnSYNI/DzrwFIDLtMXf8OCI2Co7cn7o3q4+BR8nbxb9i4uZIdX7jt5cQnYONWPJ9bt660/d9qmnz6ETZmtk2Hxg0RVlZkRZZ6ocG75uTuyo0i+0VqXAJOZp5/x8EDmLbtV558azSb5iwptrx5r65EhVxEn5dnsWzmuFa1JyGhsHGUmJSJa1X7YnWvvNSSDZuCyckpu54RrYsbusTCayzpkuKwqmr62llX88G6mg81Pl1Jjc9WY9+qfbH7sa3fGKysyIuNLLOskiSVcphGCNFcVdUzBbetyL8G/aPAOeATVVUz77T+3RJC3HWt31O9qdW0EV+8OtpkvpNrVao3qEvwwSOWiGTCXDr1Hg4VPNupO0nx8VSrUZ3FP//A5dAwosMjLBiweEKTfELwyDujOTJjbrG6rIRkNvV+idzUdJwb1afLwplsfXY4upsWeWtLz1bgzG+bOfPbZnz7dKPt8BfZ9cFcgjftwKW2Dy/88jVpMXHEnD6PQa+3TK475LtdYtAh4v7ag5qXR7UB/Wg04z1OjZtkXG5d1YXGH07nwqw5lj2EZC6bmfs/uGYjB9dspFXfnvQc8Spr3v/UuMyjbi2emDSKFSPetlyue3D7e12ntjPVvBz4dtUJ3N0rld0D3812p9Fg7VWDiA9Goq3qQY3ZK7g+YbDxcIzGuSqeE2YRu2SmZd9XSZKKKa1n5Psitz8D6gHzATvgm5JWEkKMEEKcEEKcWLGixAu9GaXExuPiWXjYxdnTnRvxCcXqGrVvyxMjhvLl2HfQ3fYtr23vHvyzOxC9zsIfVkB8bBzuXl7GaTcvTxLj73jhORNJBbXREZGcPHKMBk0aWzRfVlwClTwLv/XZe7iSVdBFD2BVyQ6nurXosXIe/bb9hGuzRnRZNAuXxg0w5OWRm5rfTZ1y4SIZkTE41qxusWwZcQk4FMlW2d2Vm/FJJdaH7gigbreOAKh6A/vnfcMvg0axZeKHWDtU4kZ4lMWyQX5PiK174bZn4+5GTkKiSY0uLQ21YHuL/mMLDg0LDxVp7O1pvuBzrixfRVrweYtmS41LoEqR/cLJw43U+MQS609t30PT7p1M6l9bNJtf35tNkgV7bIp6sq8vXy56ki8XPUlychZuboU9Ia5V7UlKzjKpb9jQjXp1q/Ldt88w77PeeFdz5LPZj1k8ly4pHq1rYe+ltqoHuuTEYjUZx/aDXo8uPpq86HCsqvkAoNhVwnv6IhJ/WUZ22DmL55MkyVRpjZGiXy96AMNVVQ0E3gJalrSSqqorVFVto6pqmxEjRpRUZnT13AU8atbA1dsLjZWWR/v04tS+IJMan0YNePXDKXw57h3Sk1OK3cejTzzG0W2WP0QDEHLmLDVq1cSrujdaKyt6PtmXg7v33tW6Do6OWFnnH6t3cq5CszatuHbxkkXzJQWH4uDjTaVqnihaLTUf9ycqsPAwVl5GJhu6Pcfmvq+wue8rJJ69wP6JM0g+H4aNsxNCyd8MKnl74uDjTUZkTEkPdc/igkOp4uONo3d+tga9/bkSaHqIrYqPt/F27S7tjA0Ora0NWjtbAHz8HkHV600GvlpC+oUQ7GpUx9bLE6HV4tGrO4lBB01qrKu6GG+7du7IzYLBrUKrpdnnnxC7bScJewMsmgsg4lwIrjWr4+LthUarpVWfHgQHmGZz9SlsODbq0p7E8PzDCbYOlRm29HO2Ll7BtVNl92G6ZVso4yduYfzELRw+Gk6PbnUB8PV15WZmHikppo2RbdvDeOW1dbw2fAOTp+4gKjqNqdMtv99mXzyPlZcPWvdqoNXi2KkXN4+bnmGXcTQQu2atAVAcnLCq5pM/xkSrpdrUuaQFbCPj0B6LZ5OkuyEUbZn8q6hKS+YkhBhAfqPFRlXVPABVVdWCq6tZhEGv5+fZ85j07WIUReHAxi1EX7pK/3HDuRYcwul9QQycPB4be3tGL5wNQHJ0HF+OeweAqtW8cPF0J+z4SUtFMqHX61kw82MW/LAKRVHY+vt6rl68xBsTxxNy9hwH9+yjYfOmfLrsKxycHOnYoxtvTBjHK72foma9urwz+yNUgwGhKPzvm29NzsKxBFVv4MRnX9Ft2RyEonDlj52kXr5Os9FDSD4fZtIwuZ37I81oNmYIqk6PajBw/JPF5KalWzRbwJyveLog2/lNO0m+fB2/MUOICw7jauBhmg/uj49fKwx5erLT0/nrgy8AsHOpwoBlc1ANKhnxieyc/rnFchXm0xM2bxEtFs9DKAoxW7aRefUatYe/TlpICElBh6j+/LO4du6IqteTl5ZOyMefAeDesxtOrVqgdXLE84neAIR8/BkZFmpsGvR6Nny6iBHfzENoFI5t3Ebc5Ws8PvZ1IoNDCQ44SMcXnqGBX2v0Oh1Zaen8Oj3/EE2nF56hag1veo18lV4j8882WzHybTKSb1gkmznHT0TRtrU3q5YPICdHx8Ilh4zLvlz0JOMnbimzxy7GoCfh2y+o/uESUDSk7dlMbsQVqr4wkuxLF7h5fD+ZJw9TqWU7ai75DQwGEn9YjCE9FYeufbBr3ArFwQnH7k8CELfkI3KuhT24/NJDryIPNi0L4k5jH4QQ3902a6qqqnFCCE/gZ1VV72Y0pvpGY7/7yVhmVp0/Qqc6Dcs7RokOXAnhl5a9yjuGWS+eyr/uwuIWFTPfhNO72OfXtfTCctDtSCAAbzfrUkpl+Zh/dj99+/1Y3jHM2rY5v2EVNqBtOScxr8HG4+UdQSo7dz+40QJ6P76wTAYq7dg56YE+j7tVWtPrayBSVdUYACHEq0KIZ4HrFFyTXpIkSZIkC3vIekZKGzOyHMgBEEJ0IX8Q649AKnf4CWJJkiRJkqS7VVrTS6Oq6q3rOQ8CVqiquh5YL4Q4VbbRJEmSJOnhVJEHm5aF0npGNKJwFE0PoOgpJA/XKyVJkiRJUpkorUHxKxAohEgEsoAgACFEPfIP1UiSJEmSZGEP29k0d3y2qqrOFkLsAbyAv9TCU28UYHxZh5MkSZKkh9JDdpim1Gerqmqx66urqipPuJckSZIkySIerqaXJEmSJP0XCE15J3igKsSv9kqSJEmS9PCSjRFJkiRJksqVPEwjSZIkSRWMvM6IJEmSJEnSA/RwNb0kSZIk6b/gIbvOiOwZkSRJkiSpXD1cTS9JkiRJ+g9QH7IxI6LwoqplpswfQJIkSZLKmHiQD9Z34B9l8tm57ff+D/R53K0H0vR6s2mnB/Ew92zJuQP0a9CyvGOUaHPYKRa06FneMcx66/RuABa36FXOScybcHoX7zbrWt4xzPribCAAY5t2LOck5i09d5DOHceUdwyzgg5+DcCKCrrdjTi9i3WtKuY++9zJ3eUdQZJK9HD1A0mSJEnSf4Eir8AqSZIkSZL0wMieEUmSJEmqYFTZMyJJkiRJkvTgyJ4RSZIkSapgZM+IJEmSJEnSAyQbI5IkSZIklSt5mEaSJEmSKhp5mEaSJEmSJOnBkT0jkiRJklTBqMrD1VcgGyOSJEmSVMHIs2kkSZIkSZIeINkzIkmSJEkVjEHzcPUVPFzPVpIkSZKkCqfC9ow06tiOZ6ZOQNEoHF6/hd2r/meyvOPz/ek8+BkMBgM5mVn8NvMLYq9cK7M8j3TuwLDp76LRKPz1+0bWr/jOZHn/116m18ABGHR6UlNSWDJtJgnRMQDMXLmUBi2bc+Hvk3w88s0yyVerQ1v8p4xBURTObtzO8dVrTJY3H/gkLQf1x6DXk5eVza5ZC0i+Eo6i1dJzxkQ8G/uiGgzs++JrIk+ctmi2mh3a0HXKGISiELxxOydW/2ayvNnAJ2k+qB+q3kBeVhZ7Zi00ZusxYyLujRugGgwEfvE1USfOWDQbQIOOj9J/yniERuHYhq0ErPrFZLnfwH60f2EAql5PTmYW6z+aR/yV69Ro2pBnP5ycXyQEu77+nuC9QRbN1rhjO56bOhFFo3Bw/Z/sum0/6P7qIDo8+xQGvZ6M5Bv874NPSY6JA6D/pNE07dIBgO3Lv+efHXssmu2WCRMH4te+CTnZeXw6+0fCwiJMltvYWPHxJ8Op5u2KwWDg4IGzLP/mDwDGv/ksrR5pAICtjTVVnB3o23uyRXJV79CGDgXbXcjG7Zy+bbu7pXbPzvSaP4MNL4wl8XwY3n6P8OiEN9BYWaHPy+Powm+JPnbKIpmK8ujQlpbv5Oe7umk7od+tMVvn3bMz7ed+yJ6XxpByPgwAp/q1eeT9SWgr2YNBZc/LYzDk5lk8o1R+5ADWCkAoCgPff4ulwydxIzaeyb+t5Ny+AyaNjb+37uLg2vw/aE39OzLg3fEsG/V2meRRFIWRH05jxmujSIqNY/76nzm2J5CIy1eMNVfOh/DWMy+Rm51NnxcGMvTdicydOAWADat+wMbWlt6DnyuTfEJR6P7eeNaPnEJ6XAIv/bKUywGHSL4SbqwJ2baXM79vAaBO1/b4Tx7NhjHTaPZsXwB+fG44di5VeGbpp/z84lhQVYtl839vPBtHTiEjLpHBv3zFlYDDJtlCt+3lbEG22l3b03nyKP4Y8x5NC7L9/NwI7Fyq0H/pbNa8OM5i2W7lGzB9It+OeJvU2ATGr1nO+X0Hib9y3Vhzcttujvy+GYDG/h146p2xrBr9LrGXrrJk8EgMej0Ori5MWreaC4GHMOj1Fsv2/Ptv8+XwidyIjefd31Zy9rb9IOLCRT4f9AZ52Tl0HvQ0T789ltWTZ9CkS3tqNPZlznND0VpbMfH7pZwPOkz2zUyLZLvFr30Tqld354VBM2ncpBZvTx7MyBFzi9X9+utuTv4ThlarYdGSCbTza8zRI+f5csl6Y82zz/lTv351i+QSikKn98azdeQUbsYlMuCXr7gecJgbRbY7ACt7O5q++DRxZy4Y52XfSGXnmzPITEjCuV4t+i6bw8+9XrBILiPIC28eAAAgAElEQVRFodXU8QSNnkJmXAI9fl5KdOAh0m/Lp7W3o94LA0gqkk9oFNp+Mo3jH3xGatgVrJ0cMegss81JFcfD1hipkM+2ZrNGJIRHkhQZjV6n45/tu2nWvZNJTdE/qtZ2dqgW/IC6Xf3mTYm5HkFcRBS6PB1BW3fSrqe/Sc3ZoyfIzc4GIPTUGVw9PIzLzhw+RpaFPwSK8mzqy42IaFKjYjDodITsCKCuf0eTmtwij29lZ2t8varWqUnE0ZMAZCXfICc9A88mDSyWzaOpL6kR0aRFxWLQ6QjbEUAd/w53zHarseFSpybhRbLlpt/Ew4LZAGo0a0RieBTJkTHodTpOb99Lk26m21rO7dtawe287Bxjw0NrY42KZbfBWrftB39v30Pz7p1Nai4e/4e87BwArp4OpoqHGwBedWtz6fhJDHo9uVnZRIVepHEnP4vmA+jUqTk7dhwF4HzwNSo72FO1qqNJTU5OHif/yf9Gr9PpCQuNwN3Nudh99ejZht27T1gkl1vBdpdesN1d3hFArdu2O4A2Y4dy+vu16HNyjfOSQi6TmZAEQMqla2isrVGsrCyS6xaXpr5kRERzMyoGVacjYmcA1W7bZwGajBlK2Pe/YcgtzOfRvg2pF6+QGpb/ZSg3NQ0MBovmk6QH7Y6NESHEOCGEa8HtekKI/UKIG0KIo0KIZmUVqoq7Gzdi443TN+IScHJ3K1bXefAzzNj+G/3fHs36OYvKKg5VPdxJjI01TifGxlHVw73E+l4DB/D3/gNllud2ld1dSS/yemXEJ+DgUbVYXYtB/Xh9y490mTScfZ8vBSAh7Ap1/TsgNAqO3p64N2qAwx2e27/LllAkWyKVPVyL1TUf1I8hW36g06RhBH7+NQCJYZdvy1YfB4/i28H9cHJ3JbXIa5cal4CjmXztBz/NlG2/0PetUWyes9g4v0azRry18Xve2vAdG2YtsFivCOTvBykm+0E8VczsB7d0eOYpzgcdASAy9BKNO/thZWtDpSpONGj7CM6elntfb3Fzq0J8fIpxOiE+BVe3KiXWV65sR8eOzTjxd4jJfA8PF6p5VeWfv0MtkquSuys3i2x3N+MTqXTb+1q1YV0qe7oRvv9oifdTu2dnEkMuYciz7CEQO3dXsuIK39usuATs3Ez32Sq+9bDzdCcmyDRfZZ/qoKp0WvoZPX5ZRoMhz1s0mySVh9J6RkarqppYcHsxsFBV1SrAFOCbklYSQowQQpwQQpxYsWLFvacSotgscz0fQWs2MKvPIDYv+IbHRg6598e56zh3lwfAv19f6jVtzIaVP5RZnmLM5itedvq3zax+8lWCFq2k3fCXADi3aTvpcYm89MvX+L8zhpjTwRb9QL3b9/LMb5v54ckhHFy0krbDXwQgeNMOMuISeOGXr+nyzmhiTp+3bLYS8pnr4Di8ZhOf932RbQuX033Eq8b5EWcvsGDAUL4cPIpuw15Ca21dptlK2u7aPvkYPk0asvu7/PEuIYeOERx0mMn/W85rcz/i6ulg9JZ+7bi3fUOjUfhw5uusW7ePmOgkk2U9erYmIOAkBoOFepdKyyUE7SeP5vD85SXehXPdmrSbOIygj8vii46Z7c5ksaDF5NGcmV/8z6yi0eDaqinHpn9KwOsT8e7eCfdHW5VBRkl6cEobM1J0ubuqqhsBVFUNEEI4lLSSqqorgFutEPXNJT/eU6gbcfFUKfItroqHG2kJiSXW/7N9N89/8DY/39Oj3L3E2DhcPT2N066eHiTHJxSra9GhHQNHD+O9l95AZ+FvUneSEZeAQ5HXq7K7GxnxSSXWh+zYR4/pE9gJqHoDgfOWGZcN/mExKeGRFs5W+G2+srsrN++QLXRHAN2mT2AXc1H1BvbPK/xjPPCHRdwIj7JYNsjvCXEq8to5ebiRFl/ytnZ6+x4GvD+p2Pz4q9fJy8rGs15tIs9b5tv9jbh4k96MKh7upJrZD3z92tB7xBAWDh1rst3tXPEjO1fk73tDP/+QhOuWeV8HPNOFp/rlH1IIuXAdd/fCQy5u7s4kJaaaXe+dd18kMjKe39fuK7asR882LJxvfoDpv3EzLoFKRba7Su6uZBbZ7qwq2eFSrxZPrZwHgJ2rC48vnsXOCTNIPB9GJXdXei2cyb73vyA9MsZiuW7Jik/ArkgPpJ2HG1kJhfm0lexxrFuLrivnA2Bb1YUOi2ZxaOIMMuMTSPj7DLk30gCIPXCUKg3rE3/spMVzSuXHIMeMmFgnhPheCFEH2CiEmCiE8BFCvAaEl7LuvxZ+LgQ3nxq4eHuh0Wp5pE9Pzu47aFLj5lM40K1Jlw4kWPAD9HYXzwZTrZYPHtWrobXS0vmJxzm6J9Ckpk4jX8bMep9PRk0kNTmlhHsqG7HBoVTx8cbR2xNFq6Vhb3+uBB4yqani412YtUs7Y4NDa2uD1s4WAB+/RzDo9SaDS+9X3G3ZGvT250rg4RKz1e7SztjguD2bauFsAJHnQnCtWR1nb080Wi0t+nTnfIDptuZaJF/DLu1JKnjtnL09UTT5V0ms4uWBW60aJEfHYinXz4Xg7lOdqgX7Qes+PTi7z/TwX/WG9Xnhw3f5ZtwUMpJvGOcLRaGSU/7YjWoN6uLdoB4XDh2zSK6NG/bz+tA5vD50DkH7z9C7dzsAGjepRUZGFklJacXWGTb8KSpVtmPJ4nXFltXwccfBwZ5z564UW/ZvJQSH4uTjjUPBdle3tz/Xi2x3eRmZ/Oj/HL/2fYVf+75C/JkLxoaItUMlen/1CccXryLuVLDFMhWVEhxKZR9v7Kt5IrRaajzuT0xA4T6ry7jJn92fZfsTL7P9iZdJPnuBQxNnkHI+jLhDJ3CqXweNrQ1Co+DaugVpRQZcS9J/0R17RlRVnS6EGAr8CtQFbIARwCbgpbIKZdDrWffpAsYsX4CiUTiycSuxl6/Sd+wbhAeHcC7gIJ1ffBZfvzbodTqy0tL533uzyyoOBr2e5bM+Y+aqZSgahd3r/iDi0mVefHM0l86d59jeQIZOmYSdvT1TluSfSZAQHcPs0RMBmPPLaqrXqYWtvT2r9+/ky/dmcvLA4Ts95D1R9Qb2zfmSZ5d9hlAUzm3aQdLl63QYM4TY4DCuBB6m5eD++Y2NPB056Rns/OALAOxdqvDMss9QDQYy4pPYPv0zi+W6lS1gzlc8vWwOQlE4v2knyZev4zdmCHHBYVwNPEzzwf3x8WuFIU9Pdno6fxVks3OpwoBlc1ANKhnxieyc/rlFs0H+e/vHp4sY9s08FI3C8Y3biLt8jcfGvk5kcAjnAw7R4YVnqOfXGoNOR1ZaBr9NnwNA7VbN8X/jRQw6HapBZePshWTeMN8r8G+zrf10IWOXL0DRaDi8cQsxl6/yxNhhhAeHcDbgAAPeHouNvR3DFnwCQHJMHMvHT0Gj1TLpx/yxN9kZmfwwdZblD3EBhw+fw699E9as/Yjs7FzmfPqTcdnq76fx+tA5uLlVYcjQPly7Fsuq76YCsGF9IFv+zP/w7dmzLXssNHD1FlVv4OCcr+izbA6KohC6aScpl6/TeswQEoPDTBomt2syuD+OPtVoNeJlWo14GYBto6eSXaSxZ4l8pz7/ks5f5++z1/7YQdqV6zQePYSU82HE3CFfXnoGF/+3ju7/WwqqSuyBY8QeKHnci/TfpD5kFz0TdzoLRQjRFohQVTW2YHoI8CxwDZipqmryXTyG+mbTTqVXlYMl5w7Qr0HL8o5Ros1hp1jQomd5xzDrrdO7AVjcolc5JzFvwuldvNusa3nHMOuLs/m9amObFj97oiJYeu4gnTuOKe8YZgUdzG9graig292I07tY16pi7rPPndxd3hH+60oZ6GNZXSb8XSaniO5f3PqBPo+7VVrTazmQCyCE6ALMAX4AUikcEyJJkiRJkvSvlTaAVVOk92MQsEJV1fXAeiGE5S9JKEmSJEkSBk2F7MAoM6X1jGiEELcaLD2AvUWWVcirt0qSJEmS9N9SWoPiVyBQCJEIZAFBkH8BNPIP1UiSJEmSZGGq8nD1jJR2Ns1sIcQewAv4Sy0c7aoA48s6nCRJkiQ9jGRj5Daqqh4xMy+sbOJIkiRJkvSwkeM+JEmSJKmCUTXlneDBeriuqiJJkiRJUoUjGyOSJEmSJJUreZhGkiRJkiqYh20Aq+wZkSRJkiSpXMmeEUmSJEmqaB6yroKH7OlKkiRJklTRyJ4RSZIkSapo5Km9kiRJkiRJD44ovMJ7mSnzB5AkSZKkMvZAT2/xm3W6TD47j8xoUSFP03kgh2n61G/2IB7mnm2/eJYGNWuVd4wShV2/xtnefuUdw6xmO/J/JeCnlr3KOYl5r5zaRb8GLcs7hlmbw04B0LNek3JOYt7uS8H07fdjeccwa9vmVwH4voJud0NP7eJkz/blHcOsVrsPA7CgRc9yTmLeW6d3l3eEiqUCHrcQQrgAvwG1gGvA86qqppip8wFWAjXI75Doq6rqtTvddwV8upIkSZIkVUBTgT2qqtYH9hRMm/MjMFdV1UbAo0B8aXcsB7BKkiRJUgUjKmZXQX/Av+D2D0AAMKVogRCiMaBVVXUXgKqqGXdzxxXz6UqSJEmSVNF4qKoaA1Dwv7uZmgbADSHEBiHESSHEXCFEqecGyZ4RSZIkSapghFI2534IIUYAI4rMWqGq6ooiy3cDnmZWnX6XD6EFOgOtgHDyx5gMBVaVtpIkSZIkSQ+BgobHijssL3GEsxAiTgjhpapqjBDCC/NjQSKBk6qqXilYZxPgRymNEXmYRpIkSZKku7EZGFJwewjwh5ma44CzEMKtYLo7cL60O5aNEUmSJEmqYIRSNv/u02dALyHERaBXwTRCiDZCiJUAqqrqgcnAHiHEWfKvz/JtaXcsD9NIkiRJklQqVVWTgB5m5p8AhhWZ3gU0v5f7lo0RSZIkSapglIfst2lkY0SSJEmSKhjlIRtE8ZA9XUmSJEmSKhrZMyJJkiRJFUxZXWekopI9I5IkSZIklasK0zPSunNHRr0/BUWjYcfaDfy+wvT6KE3btmbk9Hep7duAzya9y4Edu4zLXn9nEm39OwPw69Ll7N+20yKZOnftyvQPZ6DRaPh9zW+sWLbMZLmVtTVzFyygSbOm3Ei5wcRx44iKjDQu96pWjW27d/HlokWsXvEt1jY2/LL2N6ytbdBoNezctp0lCxdaJGvl1n5UGz0JFIWUHZtJWPuTyfIqvZ7A641x5CUlAJD05zpSdmwGwPP1sTg82gGA+F++I3W/ZX89s1qHNrR5dwxCUbi0cTvB3/1mts6nZ2e6zpvB1hfHknw+zDjf3tONfhtWceabHzn/4zqLZgN4pHMHhk1/F41G4a/fN7J+xXcmy/u/9jK9Bg7AoNOTmpLCkmkzSYiOAWDmyqU0aNmcC3+f5OORb1o8W9sunRjz/lQUjYbta9ezZvlKk+XN2rZmzPtTqePbgE8mvkPQjr8AaOH3KKPfK/zJCJ+6tflkwmQO7d5r8Ywjh7elbRtvcnL0LFh0kMtXkovVzJrZAxdnOzQaheDgOL5efgyDQaV2LWfGjfHDzlZLXHwGX8w/QFZWnkVyeXdow6MF293Fjds5W8J2V7NnZ7rNm8GfL44l6XwYrk196fDBJOPyU9/8RPi+gxbJVJRDWz+qj5mIUDQkbd9M3BrTfdblsb5UGzGOvMT8fTbxj3Ukbf+Tyi0ewXv0BGOdrU9Nrn0yg9RD+y2WrVaHtvhPGYOiKJzduJ3jq9eYLG8+8ElaDuqPQa8nLyubXbMWkHwlHEWrodeHb+PRqD5Co3D+z90cX/2rxXI9TB62MSMlNkaEENuAMaX97K8lKIrC2JnTeW/oCBJjY1m8fg1H9+4j/NIVY018dAzzp3zAs28MMVm3rX9n6jZpxNh+A7GytuaLn7/jxP4DZGbcvO9MH348i9deepnY2FjWb97Mnt27uHzxkrFm4KDnSU1NpVdXf5546inemTqViePGGZe/N+MD9gcEGKdzc3J49YUXyczMRKvV8uu6dQQGBHD65Mn7yoqiUG3sZK6+9ya6xHjqLvmOtCNB5IRfMylL3b+b6K/nm8xzeLQDtvV8uTjmVYSVFXXmLiP9xCEMmZn3l6mAUBQenTae3aOmkBmXSJ+fvyIy8DCpV8JN6rT2djR84WkSzlwodh9tJo8m+uBxi+S5naIojPxwGjNeG0VSbBzz1//MsT2BRFwu3PaunA/hrWdeIjc7mz4vDGTouxOZOzH/g37Dqh+wsbWl9+DnyiTb+JnTmTJkOAmxcSzd8BuH9uwj/NJlY018dAxfvDud54cNNVn39JFjjOr3LAAOTk78sGc7fx84ZPGMbVp7413NkWEjN+Hr68q40e2Y9M72YnVzPt9vbGRMn9qVTh1rsj/oGhPGt2fl6r85FxxHr571eO6ZJvz086n7ziUUhXbTxvNXwXb35M9fEV7Cdtfotu0u5dI1/nxxDKregJ2rC/3WfkPE/sOoesN95zJSFGqMf5tLUyaQlxCP79LVpB4KIvu2ffZGwB4ivzLdZzNO/0PoqPy/gxoHRxr/8Dtpfx+1WDShKHR/bzzrR04hPS6Bl35ZyuWAQyQXee1Ctu3lzO9bAKjTtT3+k0ezYcw0GvTqisbaih+fG47W1oYhG1YRumMvadFxFssn/f90p7bX98BfQojpQgirsgzRoHkzoq+HExsRiS5PR+DW7fj16GZSEx8VzbXQMFTV9DiaT726nD12AoNeT05WFldDQmndudN9Z2resiXXr10nIiKCvLw8tv75Jz17PWZS06PXY2xcvx6AHdu20b5jB+Oyno89RkR4OJfCLpqsk1nwIa/VatFaaYs9n3/D3rcxuTGR5MVGo+p0pAbuwrF9l7ta18anNjfPngSDHjUnm+yrF3Fo3f6+M91Stakv6RHRZETFYtDpuL4zgBr+HYrVtRw7lODv16LPzTWZX6NbBzKiYrhx+ZrFMhVVv3lTYq5HEBcRhS5PR9DWnbTr6W9Sc/boCXKzswEIPXUGVw8P47Izh4+RddMyDbfb+bZoRvT1CGIiItHl5RGwdRsde5ruF3FR0VwNDcNgKHk76tL7MY4HBpFT8Bwsya9dDfbsy28chYYmUqmSNc7OdsXqbjVENBqBVqtAQdzq3o6cC87/oDp5KpqO7X0sksv1tu3u6s4AfMxsd4+MHcq527Y7fXaOseGhsbY2ZrUke9/G5ERHkhuTv8+mBOzGqePd7bNFVenSjbTjh1FzciyWzbOpLzciokmNisGg0xGyI4C6/h1NanKLbPNWdrbGv2OqqmJlZ4vQKGhtbDDodORmlM3+8f+dopTNv4qqxGiqqq4l/4duHIETQojJQoi3bv2zZAhXT3cSYmKN04mxcVQt8gf/Tq6GhNKmSydsbG1xdK5Cc79HcfO6u3XvxMPTg9iYaON0bEwMHp4exWpiovNr9Ho96enpODs7Y2dnx/DRo/hq0eJi96soCn9s28bhf/7mYNABzpy6/2+B2qpu5CUU/kRAXmI8VlXditU5dupGvWX/w2f6p1i55v/YYvaVizi0aY+wsUHj6ETl5q2xcrv/1+8We3dXbsYmGKdvxiVi5+5qUuPsWxd7Dzeigky/3WltbWkydBBnvjHtvrakqh7uJMbevu2Z+yHKfL0GDuDv/QfKLE9Rrh4exMfEGKcT7mG/KMr/yT7s3bLNktGMXKvak5BQ+GGTmJSJa1V7s7Ufz+zJLz89T1aWjgOHrgNw7foN/NrVAKBzx5q4ulaySC5z2539bdudS8F2FxlUvFfBtWlD+q//lv7rVnD4k8WW7RUBrF3dyI0v3GdzE8zvs1U6+9NwxU/UmjEbK7fi26Wzf09S9u4qNv9+VHZ3JT22MFtGfAIOHlWL1bUY1I/Xt/xIl0nD2ff5UgAu7t5PXlY2I3evZfjOnznxw+9kp6VbNN/DQjZGTOUBNwEbwOG2fxYkis+6yx6Dfw4c5kRgEPPX/sSUhV8QcvI0er3eAomKZ7q9F0MI8zVvvjWJ71euMvaCFGUwGOjfty9d/NrTvGUL6jdocN9ZMZvDdDr9SBChQwZwafTLZJw8TvXJMwDI+OcY6ccPUXfBt9SY+jGZF86h6nX3n+kO2UzCCUGbd0bz94Llxcqaj36VCz+vR5dl+W/0hQ9f+vt8i3+/vtRr2pgNK38oszxFmXvp7na/uMXFzZXavvU5EWT5MQ8lKen1+2Dmbl4e8jtWVgotmuf/KOiiJYd4sq8vixc8gZ2dFTqdhT7072K7e/Sd0Zwws90BJJ4L4Y9nh7PlpXE0e2MwGmsLdw6bf3NNplKPHCD45WcIGfEK6f+coOa7H5gs17pUxbZ2XdJOHCnzbObe0tO/bWb1k68StGgl7Ya/BIBn04aoegMreg1iZd9XaP3qczh5e1k2n/T/0p3GjPQGFpD/wziPqKp6131tRX+iePly8zt7UYmxcbh5Ff5isaunB0nx5n4M0Lw1y75lzbL8S9+/u+Bzoq+Fl7JG6WJjY/H0qmac9vTyIj7ONFNsTCxe1aoRFxuLRqPBwcGBGzdu0KJlSx7v05d3pk3D0dERg2ogNyeH//3wo3Hd9LQ0jh0+Qmf/rlwMC+N+6BLjTb41Wbm6o0tOMKnRp6cZbyfv+APPN8YapxPWfE/Cmu8BqDHlI3KjI+4rT1GZcQlU8iz8xlfJw5WshKTCrJXsqFK3Fo+tnAeAXVUXui2axb6JM3Bt1pCavTrzyMThWDtURjUY0OfkEfqbud9m+ncSY+Nw9TTd9pLjE4rVtejQjoGjh/HeS2+gy7PMAMvSJMTG4e5V+Ifc7R73C4CufXtz8K896HWWa2A+2deXxx+rD8DFi0m4udlDwZAL16r2JCVnlbhuXp6BI8ci8GtXg5OnYoiMSuP9D/MHTHtXc6Btm+oWyWhuu8s0s931LrLd9Vg0iz0TZ5BUZPB06tVwdFnZVKlX22T+/cpNiMfavXCftXZzJy8p0aRGn1a4zyZt+wPv4WNMljt37UHqwUCwwJevojLiEnDwLMxW2d2NjPikEutDduyjx/QJ7AQa9unOtUPHMej0ZCXfIPpUMB5NGpAaFVPi+pIEd+4ZmQ4MVFV16r00RCD/J4pVVW2jqmqbESNGlFofdvYc1WrVxKO6N1orLV2f6MORPQF39ViKouBQxQmAWr4NqO1b3yID9c6ePk2t2rWoXqM6VlZWPPHUU+zZZdodunf3LgY8mz9IsHffvhw+lP+4Lw58nu6dOtG9Uyd+WL2ab5Yu5X8//IiziwsOjo4A2NjY0KFTR64UGYz4b2WGXsCmWg2sPLwQWi1OXXuRdiTIpEbrUtjN6ujXuXBwq6KgccjPZFu7Hra165H+97H7znRLUnAoDj7eVK7miaLVUvNxfyICDxuX52Vk8nu359jY9xU29n2FhLMX2DdxBsnnw/jr9beM8y/8vIFzq361aEME4OLZYKrV8sGjejW0Vlo6P/E4R/cEmtTUaeTLmFnv88moiaQmp1j08e8k9Mw5vGv64FndG62VFf5P9OXQnn33dB/dn+pr8UM0W7aFMn7iFsZP3MLho+H06FYXAF9fV25m5pGSYtoYsbXVGseRKIqgbevqRESmAuDkZAvkfxkf/Hxztu2wzAd+YnAojkW2u9pmtrs13Z5jXd9XWFew3d1qiFSu5onQ5P9prOTljlPNGmREx5b0UP9KZugFbLxrYO2Zv886+/ck9VDJ+6xT+87FBrc6d+9l8UM0ALHBoVTx8cbRO/+1a9jbnyuBpn9Tq/h4G2/X6dKOlPD8swjTY+Op8WjL/Px2tng1a0Ty1fv/cij9/1diz4iqqp0fVAiDXs+yjz7lk9XfoNFo+GvdRsIvXeaVCWMJOxvM0b0BNGjWhA++XkxlRwfadevKy2+OYVTfAWi0Wub9mt9tnpmRwdzJ0zBY4JuCXq9n1owZrPrxRzQaDevWruXSxYu8+dYkzp05y97du/n9t7XMXbiAXYEBpN64waRx4+94n+7u7ny+YD6KoqAoCtu3bCVgrwVOtTToif56HrVnL84/tfevLeRcv4r7K8PJuhhC+pEgqvZ/Hke/zqh6Pfr0NCLnfwyA0GipMy+/98qQeZOIL2aCwXLftFS9gWOffUWPZXPyT+39Yyepl6/TYvQQks6HEVnkA6I8GPR6ls/6jJmrlqFoFHav+4OIS5d58c3RXDp3nmN7Axk6ZRJ29vZMWTIXgIToGGaPngjAnF9WU71OLWzt7Vm9fydfvjeTkwcs85wMej1ffjSbz75bgaJR2PH7Rq5fvMyQCeMIOxfM4T378G3WlJnLFlPZ0ZH23f0ZMmEsw/r0B8DDuxpunp6cOVo2ZyIBHD8RRdvW3qxaPoCcHB0LlxR+aH256EnGT9yCra2WD9/vhpWVBkURnD4Ty7bt+Y0O/y61eLJvQwAOHg5n1+5LZh/nXql6A0c++4peRba7G5ev07Jgu4u4w3bn3qopzV4fhKrToxoMHJmzhJwbaSXW/ysGPZFfzqfuZ4sQikLSji1kX7+K55DhZIZdIO3wAdwGPI9T+06g16NLT+P6F58YV7f28MTKzYOMM/d5Jp4Zqt7Avjlf8uyyzxCKwrlNO0i6fJ0OY4YQGxzGlcDDtBzcHx+/RzDk6chJz2DnB18A/F979x97VV3Hcfz5khpBjlmhU4aTAoxSFxitHzZdhlNYsWgmqRNwM8vm2irRypUoubUssiCXacpgDmEISaWUbpGWbE3hi0AYQkpr/RKrFUYW+O6Pz/m249d77/dL+97z+Xz5vh7bHfee87nc1/d87j33cz6fzzmXnnvv5/ybFjJv3Z0IsfP+H7P/6WcGPeNwUPL8jm7QYJzN0Y+YOfmMbr/G/+XBp7dz6ikTcsdoa/e+Z9l+wbtyx2jpjI1pnHrl1PMyJ2ntsp6HmH3q1NwxWtqwO01anjHptMxJWnt4z05mzV7Rf8EMHtgwD4Dlhb7vFvQ8xNYZg3c22mCa9nBqgBFEGIgAAAcaSURBVC1524zMSVr7zLbBvb5RF7Sa6NM1s+7Y2pUv5wc+Nq3Rv2Oghlnby8zMzEpTzBVYzczMLBnh36YxMzMza457RszMzAoz3CawujFiZmZWmOHWGBlmf66ZmZmVxj0jZmZmhRkxzLoKhtmfa2ZmZqVxz4iZmVlhjiny0mTd454RMzMzy8qNETMzM8vKwzRmZmaF8QRWMzMzswa5Z8TMzKwwvuiZmZmZWYMU0fVfBhxePz1oZmZHo0ZPtp23ZktXvjtXXHRmkScNN9EYGVSSroyI7+bO0U7J+UrOBmXnKzkblJ2v5GxQdr6Ss0HZ+UrOZq80FIdprswdoB8l5ys5G5Sdr+RsUHa+krNB2flKzgZl5ys5m/UxFBsjZmZmdhRxY8TMzMyyGoqNkdLHAEvOV3I2KDtfydmg7HwlZ4Oy85WcDcrOV3I262PITWA1MzOzo8tQ7BkxMzOzo0hxjRFJcySFpCm1ZZMl/VDSXklPSPqppLOrdQskPSepp3Z7a5eyXS9pp6Qnq9d5p6RNkn5de+21VdlvSfpin+d+uxu5aq9xuMqwTdIWSe+plk+otuniWtmxkv4jaVn1eJGka7qZb4A5D1brfiXpO5Iaf4+2qeerJe2ptuPYpjMNIN891ftwh6S7JL26sHzfq+r7SUlrJR1bSrbauqWSDuTI1SmfpOWSnqntY6YWlE2Sbpa0W9IuSZ/Kka2WUZJ+LmlmbdlFkjbmzGUDEBFF3YA1wKPAourxa4DdwOxamdOBBdX9BcCyBnK9G9gMjKwejwXGAZuA6S3KjwF+A7wJeCPwDHBclzMeqN0/H/hZdX8CsBfYWlt/FdDTu+2ARcA1DdVxp5w7qvuvAh4BPtzw+69dPU+r8j0LjG0y0wDzzSJdlEnAKuCqwvKNqZVZAnyulGzV/enAyvp7s5R8wHLgwly5+sl2ObACOKZafkLOnFWG04FdpO+O1wJPAxNz5/Kt862o36apjpbOAt4HbCB9QV4KbI6IDb3lImIHsKPheCcB+yPixSrD/ipzy8IR8XdJ1wPLqkVfioi/NRG0Mgb4a+3xQWCXpOkR8Tgwl9TwG9dgplb65gQgIg5JegyY1HCelvUM/B7a13eDOuYDkPRLYHyGbNA+H5COXIFR5Lkyc7vP8AjgFuASYE6GXL2OaB/TsHbZrgIuiYiXquV/zhcxiYgdkn4AXEdqjKyIiL2ZY1k/Shum+RCwMSJ2A3+RdCZwGrCln+fN7TNMM6oL2X4CnFx1R94m6Zzauntqr31L78KIWAW8jnRUuLILmfoaVWV4CrgTWNxn/b3ARyWNBw5T+wJrWH85kTQaeD+wveFsneq5BB3zVcMzlwG5uqXb5pN0N/BHYAqwtKBsVwMbIuIPGTLVdarbm6vhkW9IGllQtomk/e/jkh6UNDlDtlZuJDUuZwJfzZzFBqC0xsjFpC9Mqn8v7ltA0vpqXHxdbfHqiJhaux0c7GARcQB4O+mqfs8BqyUtqFZfWnvthbWs44ETgXENjZEfrDJMAS4AVujlh1UbgfNI23V1A3na6ZRzoqQe4BfAjyLiwSaD9VPP2Q0g323AIxHxaIZ4HfNFxOWknrhdpJ65ErJ9AfgIeRpHL9Nh232e1IB7B/B60hF/KdlGAv+KiOnAHcBdTWdrJSJeIO3jVvb25ljZihmmkfQG4FzgdEkBjCB15d4InN1bLiLmSJoOfK3pjBFxmDRHZJOk7cD8fp7yTdJQ01uAG4CFHUsPoojYrDTR8vjasn9LegL4LKnH6YNN5WmnRc69EZFlgl4tU6t6Xp4zU127fJJuIG3Hj2eM13H7RcRhSatJn4W7C8i2Cnge2FO1h0dL2hMRTQ8Ptss3PyKWV6tfrHqXGploPpBswO+A+6oi68lQpx28VN1sCCipZ+RC0tjeKRExISJOJk363A2cJWl2rezopsNJenOfLsipwL4O5WcCJ5Amdy0G5qhLZ/m0ef0ppAbd831WfR24LiL6Ls+iQ84sjrSem9Yun6QrSJOBL+4dv8+hTb7fSppUrRepEfxUIdluj4gTq33OBOCfuRoiHer2pGq9SEPZTc+X6/S5+D7pIBLgHNL+2uyIFdMzQho6+EqfZfeRxv0+ACyRdCvwJ+AfwJdr5eZKem/t8Scj4rFBzncssFTSccAhYA+py3Itac5I79DQ/irvraQZ8AG8IOla0mTWc1/xPw+eUdUQB6SzKuZXR6L/KxARO4GdXcwwEP3mzKhlPSudsngtadjtSUkPRMQVpeQjzcXYB2yutuO6iLipkHyfANZLGkOq722ks7lKyFbSj6m1y7dG0vGkbddD2p6lZDtE2v99GjgA5PhM2FHAV2A1MzOzrEoapjEzM7NhyI0RMzMzy8qNETMzM8vKjREzMzPLyo0RMzMzy8qNETMzM8vKjREzMzPLyo0RMzMzy+q/uQ7qp9DOfCkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "corrs = df.corr() # calculate the correlation table\n",
    "# as this is a symmetric table, set up a mask so that we only plot values below the main diagonal\n",
    "mask = np.triu(np.ones_like(corrs, dtype=np.bool)) \n",
    "f, ax = plt.subplots(figsize=(10, 8)) # initialise the plots and axes\n",
    "# plot the correlations as a seaborn heatmap, with a colourbar\n",
    "sns.heatmap(corrs, mask=mask, center=0, annot=True, square=True, linewidths=.5) \n",
    "# do some fiddling so that the top and bottom are not obscured\n",
    "bottom, top = ax.get_ylim() \n",
    "ax.set_ylim(bottom + 0.5, top - 0.5);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to do some simple modelling, to test the feasability of models. Here we will do this with linear regression. To perform regression modellings we now create a matrix of independent variables (features) given by `X`, and the dependent or target variable `Y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop(['Y'],axis=1) # drop Y from our dataframe\n",
    "Y = df['Y'] # create a dataframe with just the Y values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we split the data into training and testing sets, and model the data using the  `sklearn` Linear Regression model. Recall the method `fit` trains the data, then `predict` uses the independent test variables to predict the target values. This can be compared against the dependent test variables to calculate $R^2$, the square of the correlation coefficient. The method `score` calculates $R^2$ for the training set. This will typically be greater than the $R^2$ for the testing set. As is apparent the fit is not fantastic, but we will see if it can be improved using regularization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training score is 0.4681\n",
      "Testing score is 0.4711\n"
     ]
    }
   ],
   "source": [
    "rng = np.random.RandomState(1) # make sure the results are repeatable\n",
    "# split into a training set with 80% of the data, and a testing set as the remainder\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X,Y,test_size=0.8) \n",
    "linear = LinearRegression() # instantatiate the linear regression model\n",
    "linear.fit(X_train,Y_train) # fit the data to the model\n",
    "training_score = linear.score(X_train,Y_train) # calculate rsq for the training set\n",
    "# use the independent variables for the testing set to predict the target variable\n",
    "preds_linear = linear.predict(X_test) \n",
    "# calculate the correlation of the predicted and actual target variables\n",
    "rsquared_linear = r2_score(Y_test,preds_linear) \n",
    "# print the training and testing scores\n",
    "print(\"Training score is\",round(training_score,4))\n",
    "print(\"Testing score is\",round(rsquared_linear,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also evaluate the model graphically by plotting the predicted test values against the actual test values, and plotting the line of best fit for this data. This is compared against the line $x=y$, which is shown in black, and will alway have a greater slope that the line of best fit. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeXgV1dnAf2/CJSRsYReCCCL7vihatQpWQXGJgCJqlbaWWsUFLR9YN6hQULSK2mqhCi5UAbGIQgUV1IKiLGERBZWdsCqELdtNcr4/Zu7l5mZm7tw1C+f3PHlyZztzZm5y3nPeVZRSaDQajUYTSFJ5d0Cj0Wg0FQ8tHDQajUZTBi0cNBqNRlMGLRw0Go1GUwYtHDQajUZTBi0cNBqNRlMGLRyqMCJysYhsKe9+VAVEpIWInBCR5PLui0aTCLRwqAKIyA4R+VXwfqXU/5RS7cqjT8GIyDgR8ZoDbI6IfCEiF5R3v9yilNqllKqllCqO531E5FIRKTHf03ER2SIiv4nnPS360FJElolIrohstvrbsrimvogcEpHlAftuMZ/D95MrIkpEepnH+5r3OSoiO8Lph4gMF5HioPYvDTi+Q0TyAo4tifa9nG5o4aCJOSJSzebQbKVULaAhsAyYm+D7Vxb2mu+pDjAKmC4iiRTybwFZQAPgYeAdEWkU4ponge8CdyilZpkCtZb5PHcB24C15ikngVeB0RH248vA9pVSnwZdf03AsStC9F8ThBYOVRhzFronYHuHiPxJRDaYs7XZIlIj4PjVIrIuYGbfNeDYWBHZas5mvxWR6wOODReRFSLyrIgcBsY59UspVQTMAjIC/9lD3L+niGSZ959r9n1C4HOKyBgR2Q/McNHeGBHJDpidX2buP09EVovIMRE5ICJ/M/e3NGe91cztZiKyQEQOi8iPIvL7gLbHicgcEXndbH+TiPR2/cWdek9KKbUIOAx0NdsW8z0fNL/DDSLS2TzWwOzTMRH5WkSeCJzJu0FE2gI9gceVUnlKqXnARmCwwzUXAJ0x37sDtwOvKzMtg1Lqa6XUGxgCI+p+aGKLFg6nHzcCA4BWGAPOcDAGX4xZ3B8wZmr/BBaISIp53VbgYqAuMB54U0SaBrTbB+OfvDEw0akDIlIduA34GTgS6v7m+f8BZgL1MWaU1wc1e4Z57CxgRIj22gEjgXOVUrWB/sAOs52pwFSlVB2gNTDH5jHeAvYAzYAhwF99AsbkWuBtIB1YALzo9E6sEJEkEbkWY6X1o7n7CuCXQFuz7aEY7xHg70A+0BT4rfkT2N4GU1Ba/fzDPK0TsE0pdTzg0vXmfqs+Jpv3HQnY5uIRkbPMfr/u7uld9aOHiPwkIt+LyKMWK8ZZYqi6lohIN5f31Zho4XD68bxSaq9S6jDwPtDd3P974J9Kqa+UUsVKqdeAAuB8AKXUXPO6EqXUbOAH4LyAdvcqpV5QShUppfJs7n2jiOQAeeb9hpiriFD3Px+oZvbdq5R6F/g6qO0SjFlmgXl/p/aKgRSgo4h4lFI7lFJbzXa8wDki0lApdUIptTL4IUTkTOAiYIxSKl8ptQ74F/DrgNOWK6UWmTaKN4BwBqdmAe/pP8ADSqmsgP7VBtoDopT6Tim1zxykBwOPKaVOKqW+AV4LbFQp1VUplW7zc5d5Wi3gaFB/jpr3tOJe4Cul1JoQz3Qb8D+l1PbQj++qH59jrFYaYzz3MEqrp24BWmJMFpYBi0Uk3eW9NWjhcDqyP+BzLsY/IRj/RA8GziaBMzFmxojIbQEqmhyMf8yGAW3tdnHvOUqpdKAJ8A3QK+CY0/2bAdk+dYTN/Q4ppfLdtKeU+hG4H0P9dVBE3haRZuZ1v8OYlW8WkVUicrXFczQDDgfNancCGQHbwe+5hsXM1o695nuqAzwP9PMdUEotxViF/B04ICLTRKQO0AhDgAa+l50u7xfICfO+gdQBjgefaL6zezHsAaG4jSBhFU0/lFLblFLbzcnKRuAvGCs4zOMrTHVUrlJqEpCDsfLVuEQLB42P3cDEoNlkmlLqLVMlMB1DddDAHLi+ASTgetfpfZVSP2Goe8YFqKZs7w/sw7BPBN7vzOBm3T6P2Yd/K6UuwhAiCsOgilLqB6XUMIwZ6ZMYRtCaQW3vBeqLSOBsugWQ7fYduEEpVQCMAbqISGbA/ueVUr0wVCxtMWbMh4AiSr+XFoHtmbaPEzY/L5unbQLODnq2bub+YM7DUGF9a9p6pgLnich+CXD5FZELMQTqO2E8fjj9AOM7FJtjbo5rgtDCoergEZEaAT/heuxMB+4UkT6m0bOmiAw0/zlrYvxzHQIQw7WyczSdVUptBhYD/+fi/l9iqIJGikg1EbmO0iqtsJ5HRNqJSD/TnpKPob4pNp/tVhFppJQqwZht4jsW0PfdwBfAJPNdd8VYccxy8+wiMlNEZro5VylVCDwDPGZee675TB4Mb598oNhUX72LIXDTRKQjhgE4sK1OQd49gT93mud8D6wDHjef7XoM29Q8i+79F0N10938eQzDu6i7Ku3yezswL2il5bOp1AA8xqbUMO1LIfshIleKSBPzc3vgUeA9c7uFiFwoItXNa0djrHJXuHnnGgMtHKoOizAGOd/PuHAuVkqtxtDTv4hhJP4R01itlPoWY4D6EjgAdCE2/2hTMIzHjUPcvxAYhDEA5wC3Ah9g2BDCfh4Me8Nk4CcM9U9j4M/msQHAJhE5gTETvilIXeVjGMbAuBfDLvC4Uuojl899JuG9v1eBFiJyDYZqZbr5TDsxjNFPm+eNxFAT7scw3ofyHrLjJqC3eY/JGLYh38TgFhHZBMbKRim13/eDYRPwmp8xz6+B4QRhpVL6Jcbf6iKMVU4eEBiPYNsP4DJgg4icNK9/F/ireaw28JJ5XTbGd3qlUupnNK4RpYv9aCohIvIV8LJSKtIBsFwwZ8brga5KKW+c7zUcuMNUn2k0YaFXDppKgYhcIiJnmGql2zFUDB+Wd7/CRSlVqJTqEG/BoNFES2WPJNWcPrTDiDmohRFzMUQpta98u6TRVF20Wkmj0Wg0ZdBqJY1Go9GUoVKrlRo2bKhatmxZ3t3QaDSaSsWaNWt+Uko5JlOMm3AwXdg+x3AbrAa8o5R63PTvvoRTofHDlVLrzACnqcBVGBGlw5VSa8u2fIqWLVuyevXqeD2CRqPRVElEJGT0fDxXDgVAP6XUCTNgZ7mI/Nc8NlopFRwteSXQxvzpg+Gn3CeO/dNoNBqNDXGzOSiDE+amx/xxsn5fh5nO10x2li6ls35qNBqNJkHE1SAtIskisg44CHyklPrKPDRRjPTBz8qplNAZlE4atofSicw0Go1GkyDiKhzMVMndgeYYCbk6Aw9hpBs+FyP//hjzdKukWGVWGiIyQoxiLKsPHTpkcYlGo9FooiUhrqxKqRzgU2CAUmqfqToqwMj94kugtofSGSWbY+StCW5rmlKqt1Kqd6NGoSoXajQajSYS4iYcRKSRmMU1RCQV+BVGjvym5j4BMjFSP4NRLes2M4Pm+cBRHQGr0VR+5mdlc+HkpbQau5ALJy9lflZMM5tr4kQ8vZWaAq+Zed2TMAq9fCAiS8WoGywYKXnvNM9fhOHG+iOGK+tv4tg3jUaTAOZnZfPQuxvJ8xoZvLNz8njo3Y0AZPbQJsWKTNyEg1JqA9DDYn8/i9Mxq3zdHa/+aDSaxDNl8Ra/YPCR5y1myuItWjhEiNfr5W9/+xu33norGRnxe4eVOkJao9FUbPbmWJcTz87Jo9XYhTRLT2V0/3ZaULgkOzubzp07k5OTQ506dfjjH/8Yt3vp3EoajSZuNEtPtT2mOKVm0naI0EyZMoVu3bpRWFjIyJEjufPOO0NfFAV65aDRaOLG6P7tStkcrNBqJmfy8vK47rrr+Ogjo9Dg5s2badeuXdzvq4WDRqOxZX5WNlMWb2FvTl5EKiDfub427FIk2KmfTneWL1/OXXfdxcaNG2nbti2rVq2iTp06Cbm3Fg4ajcaSWHkaZfbI8J9/4eSlZFsIAif10+mIUooLLriAr74ykkosWrSIK6+8MqF90DYHjUZjiZOnUaSM7t+OVE9yqX2pnmRG94+/mqSycPjwYYYMGeIXDFlZWQkXDKCFg0ajscFO1RONCiizRwaTBnUhIz0VATLSU5k0qIu2N5hMmjSJBg0asGDBAp566im8Xi/du3cvl75otZJGo7GkWXpqXFRAgWomjUFRURETJkxg/PjxAKxYsYLzzjsvxFXxRa8cNBqNJVoFlBiWL1+Ox+Nh/Pjx3Hrrrezdu7fcBQNo4aDRaGzQKqD48/zzz3PxxRcDMGPGDN544w2aNq0YZWy0Wkmj0djipAKK1s31dObkyZOMGjWK6dOnA7B48WKuuOKKcu5VabRw0Gg0YaMT6kXOZ599xqWXXgrA2LFj+ctf/oLH4ynfTlmg1UoajSZs7Nxcx7+/qZx6VPFRSvHCCy/Qv39/AMaNG8ekSZMqpGAAvXLQaDQRYOfOeiTXy/ysbL16CGLz5s106NABgIEDBzJjxgwqerEyvXLQaDRh4+TOGk2QXFXkk08+8QuGCy64gPfffz8qwfDEEyACc+bEqofWaOGg0WjCxsmdVedJMsjNzeXGG2/k8ssv5+yzz2bWrFl88cUXGEUww+enn+CWW+Cxx4ztzp1j2FkLtHDQaOJIVS2Rmdkjg/RUa125zpME27Zto2bNmsydO5dhw4axYcMGbr755ojaUgoeeQQaNYJ//xvGj4eCAujYMcadDkLbHDSaOFGZPXrcuKmOu7ZTmXTcdkFyge3VTfUgAjm53irpAvviiy/y5z//mRo1ajB06FBmzpwZcVv79sHIkfDuu8b2okWQqDRLWjhoNHGispbIdCvUgtNx2w30we3l5Hn9x+IhMMsr/uKnn37y2xI6d+7MBx98wFlnnRVRW0rB5ZfDJ58Y208+CQ88ANUSOGJr4aDRxIl4JK5LBOEINTd5kqzac9N2IG4H/PJara1Zs4ahQ4f6t1evXk1KSkpEbe3eDXfccUowfPwxXHZZLHoZHtrmoNHECTvde0XXycdaqLm5zukc34CfbRYLciotGo80404UFxfTokULevfuTWFhIcuWLUMpFZFgKCmB88+HFi1g+XJ4+GEoLCwfwQBaOGg0caOyJq6LtVBzc50CW4N9OAN+Ildr+/fvZ8CAAezevRuAdevW+SOfw2XbNvjVr8As4cDGjTBhApRnfJwWDhpNnKisietiLdSs2rPCbkUQzoCfqNXaqFGjaNq0KcuXL+fpp5+mpKSE+vXrh92O1wtt2kDr1rB6NTz1FBQXw9lnx7S7ERE3m4OI1AA+B1LM+7yjlHpcRFoBbwP1gbXAr5VShSKSArwO9AJ+BoYqpXbEq38aTSKojLUL3BqaI23P5610JNdb5lwr+0M4dSVG92/n2oMqEgoKCmjZsiX79+8HDNtCp06dImpr82YwY+MA2LQJzjwzFr2MDfE0SBcA/ZRSJ0TEAywXkf8CDwDPKqXeFpGXgd8BL5m/jyilzhGRm4AngaF2jWs0mvgRa6Fm1V6rsQtRFucGrwjCGfBjLdgC2bJlC8OGDfMLhsOHD1OvXr2w28nPh0cfhRdeMLYvvxwWLzainisScRMOSikFnDA3PeaPAvoBvmiQ14BxGMLhOvMzwDvAiyIiZjsajSYCQnn5lGfabbcrgnAH/FgLNqUUt99+O2+88QY1a9Zk/vz5XHfddRG1tXEjdO1qfL7+evjHP+CMM2LW1ZgSV1dWEUkG1gDnAH8HtgI5Sqki85Q9gO9bzAB2AyilikTkKNAA+CmozRHACIAWLVrEs/saTaUmlFtneQfphbsiKA/13NGjR7nzzjt5++23AaN8Z7du3cJu58QJI05h5kxIToYbboC33opxZ2NMXA3SSqlipVR3oDlwHtDB6jTzt9WiqsyqQSk1TSnVWynVu6JnNdRoEoVVmo5QXj6JdvsMpqIb7J977jnS09N5++23mThxIkVFRREJhjlzoHZtmD4dhgyB/fsrvmCABAXBKaVyRORT4HwgXUSqmauH5sBe87Q9wJnAHhGpBtQFDieifxpNZcZuBWAXeObT6VeEIL2KaLAvLi5m8uTJPPLIIwC8+eab3HLLLWG3k58P48YZ0c0AI0bAP/8Zw47GmbitHESkkYikm59TgV8B3wHLgCHmabcD75mfF5jbmMeXanuDRhMauxVAso2F06fTr6xBevFk1apV9OnTh0ceeYQ+ffqwc+fOiATDG29AaqohGH73OzhypHIJBoivWqkpsExENgCrgI+UUh8AY4AHRORHDJvCK+b5rwANzP0PAGPj2DeNpspgN9MvVqpMfIFgrCwunLyUvu0bVcogvXjx/vvvc95557FmzRpeffVVvvzyy7DtmidPwn33wW23GdsvvAD/+hekp8ehw3Emnt5KG4AeFvu3YdgfgvfnAzfEqz8aTVXFzusnw/TqmbJ4C9k5eQinjHjZOXnMW5PN4F4ZLNt8qFy8lSoKOTk5PPDAA8yYMYOUlBTeeecdrr766rDbeeQRmDjR+DxyJEyaBLVqxbizCUQn3tNoKjlOXj8+nf6Fk5eWESB53mKWbT7EirH9Et3lCsP8+fO5/vrrAXjggQf461//GnZepKNHYfRow+AMhvpoxIhY9zTxaOGg0VRy3MQBVATjc0VCKcW0adO48847Abjvvvt45plnwm7n3nsNtVFBAdx4oxG30KBBrHtbPmjhoNHEkUQFmYXy+nETcFaeAXGJZOvWrZxzzjkAXHHFFcyYMYNmzZqF1cbhw3D//YbhGWDlSujTJ9Y9LV+0cNBo4kQigszcRkAH2xygtPG5vAPiEiWYPv/8cy655BIAbrrpJmbNmkVSUnh+ObfdZgiFatXgD3+AKVOMOIaqhhYOGk2ciHcluHAjoBX4BURG0AAcaV9jMagnQjAVFBTQuXNn/6ph+vTpYafXPngQmjQ5tb16NUQQE1dp0MJBowlBpANgvPX8oQZ0q+M+wRBshI6kr7Ea1OMtRHfu3EnLli0BaNq0KWvXrqV2GFN9pWDaNKP4DkDdukZt59QqHg6i6zloNA6EU4UsmHgHmYUa0ONdByFW6TfiKUTnzp1L9+7dAbjqqqvYu3dvWIIhOxuSkuDOO42aC5s2QU5O1RcMoIWDRuNINANgvCvBhRrQ7Y4niZTKwRRJX+dnZVsauMEQoG6EZ3B/3e636ktwXqmTJ08iItx44420bduWH3/8kYULF7ruU0mJUXjHV6qhVSujdGfHjq6bqPRo4aDROBDNrDbeieVCDeh2FdiKlSqzCsrskcHgXhn+lBvJIvRsUZcpi7eUESS+1ZQTbldXAH3bWyfQtNsfiNXKbtRL79Guc3f/OcuXL6d169au+gKG51FyMowZA126wA8/GGU8y7NkZ3mgbQ4ajQPhVCGzIp6J5ULFNwQfTxKhOChdWeAqaN6abP/xYqVYsfVU3stAe4LVaiqYcGwGyzYfCmt/IIF9UaqEvdP/QNGRfXhqN+Djjz/msssuC9mGj5ISI07hnnuM7a5d4dNPDUFxOqKFg0bjQLzLTkZLKOETeLzVWGu1yt6cvLAGfLe2gGjPc3O975zi3KPsnzWGoiP7ADjj9qkhBUOgo0GtI2fwzbReAFxxBbz8sqFKOp3RwkGjcSCeZScTjdMqyO1AbmdnsCI9zZ0eJprVWbP0VDYvncvRL+dQnHeMWl2voP6AkTSvV9PxOp86KregmOOrW7FjmVFqZsANJ1g0u1aFK9lZHmjhoNGEoCLWHIgEp1WQL1AulrhNuB/p6szr9ZLyxcsc/mguAE1/8zzVG5/t6topi7fw04bG/LSgp3G/Nvupf/k3HDszCZHTN9dUIFo4aDTlSCJTVvjaHbdgEzl5XgBqeAyfFKsBOlqOmvdw269w3sOqVasYOXIkX3/9NY3OyKDl757nUFGKq2u9Xvh2YQY5yw0BUvvcbdTr+x0isDcnjAes4mjhoNGUE1ZBZKPnrmf8+5vIyfXGTVgUFJX4Px/J9fLQuxuZNKgLkwZ1KTVA923fyJ/OO5KqW+G4ooYjGDIzM3nvPaNG2Ny5cxkyZIjtucG8+io8/jjk7GlH9TNyaDBwPdUbngi7z6cDWjhoNFES6ezfygjsLVEcyTVm3G4ijsO9t1Pcxoqx/WyvtUr57YRbo304UdbHjx9n5MiRfsHw+eefc/HFF7vqT0EBPPHEqXoLY57+mfdyVlVYR4OKgI5z0GiiIJoIajdGYKeAu0juHalnkF3MhB1u4zncBhlOmzaNOnXq8Oabb/LYY49RUFDgWjBMnWrkRJo40fBE2roVJj/YIK4xKFUBvXLQaKIgmrxAdl46wdgN3Hb3fnDOesB6tRGpZ5CVXSC3sMi/ygkkIz3V9SBr92y+UqYPXt6GbctmM3r0aACWLFniOnYhNxfOOgt++snY/vBDyGucza/nVH7Ps0SghYNGEwWRzsTnZ2VzsqDI1T3sBm6n2tF2qhm3nkF26qrgdODRxoA4Ccjt329i8EOGIBg0aBDPPvus65rOH39s5EPyCYY9e2DVwfJNS17Z0GoljSYKIskL5BtUc4K8edI8SXiSSzvYOw22TvewU0e5SenhVl0Vi/Qgo/u3wyqk4OS3n7Fvxr0AtLzqTt555x1XguH4cahTBy6/3Ih4XrbMcKnNyIhdosDTBb1y0FR6yrOCWSQ++nbRyPVqpvhjDtw8Syj3U7uVRai4jXBUZdHGgGT2yOD+2ev826qokCOfvcbx1YbRueF1Y6H9RYiLqLT33jPKdh4/bmyvWQP16p06rkulhocWDppKTXlVMAsUSOlpHlKqJXE0z537qZOePZzB1nfeg3PWl8mZBJG7ZSZqEPW9Qx+FB7f5Vwu1ew4k/dLfkuRJCfkcu3YZtgWAc86BL76ACy4oe160ebJON+KmVhKRM0VkmYh8JyKbROQ+c/84EckWkXXmz1UB1zwkIj+KyBYR6R+vvmmqDuWhKghWuxzJ9VJQVMKzQ7s7uoP6sBuMxGw7HDJ7ZPDMjd1imho83nUoIOgdKsWJDR+x/03D6Fyrx0DqX/5HkjwpIZ/jvfegc+dT22vWWAsGiH8K9apGPFcORcCDSqm1IlIbWCMiH5nHnlVKPR14soh0BG4COgHNgI9FpK1SKnYhm5oqR3moCsJRu1ipvEb3b8eo2evKBJYps+1wVzyRRBhb9cvXRqh607HA9w6Ljh0k+6XfApDSoisNr36AlDqNKFaqTCnTQLZuNVYJYGRPfekl+MUvnO9ZlfJkJYK4CQel1D5gn/n5uIh8Bzh9C9cBbyulCoDtIvIjcB7wZbz6qKn8lIeqwK1AslN5TRrUxTbiOFKhFo46yi4yGwFvsdEzp3rTsWBvTh4F2d/5VwvJdRrTZOgTJCUls3XSVbbXKQVz58Lvf29sX3IJLFkC1au7u29VyZOVCBJicxCRlkAP4CvgQmCkiNwGrMZYXRzBEBwrAy7bg4UwEZERwAjAtVubpupSHim13QokpxVGhk0bdVNjV1HGzlBvF5kdTHC96VgZ/r1eLyeXTOXQuk9IrtOIuhcMpXb3AYCzUM/Kgp5GnjzOPdeo69y9u+3pmiiJuyuriNQC5gH3K6WOAS8BrYHuGCuLZ3ynWlxe5i9WKTVNKdVbKdW7UaPQlaI0VZt4V1uzwq3u2snwPLp/OzxJZf/kTxYWhW13sMLJHTWc1Ynv3GgiwQPJzs6mevXqHMr6iLSW3Wj22xf9gsFOqCsFM2eeEgxXXGEYnbVgiC9xXTmIiAdDMMxSSr0LoJQ6EHB8OvCBubkHODPg8ubA3nj2T1M1SLSqwK3u2m6F4RMJtWpUKxNh7C1WEdkdgrFbtdw/ex3JFhXh7PDVZIgmEtzHjBkz+NOf/kSNGjUYMGAAtz36Ak8v+d7xHa5YAfffD6tXQ+PG8MorcPXVrm4HlF3tBCYT1DYHZ+ImHMRwTH4F+E4p9beA/U1NewTA9cA35ucFwL9F5G8YBuk2wNfx6p9GEw1uBJKT4TnQtz+YWBjTndqwEgyeJKEEKA5SL53IL3Jcbbjp64kTJ6hduzYAzZs358svv6Rt27YAXN+zueU1JSUwfboR5Qzwwgtw112QFIauw8q28ubKXf7jOkLamXiqlS4Efg30C3JbfUpENorIBqAvMApAKbUJmAN8C3wI3K09lTSVmcweGXFNdR1tG8kiflXclBu6UTul7FzRW2KsZCJ1b920aRMXBPiWfvvtt37BYMdHH0GHDoZgaNMGli6FkSPDEwwQXq1rTVniJhyUUsuVUqKU6qqU6m7+LFJK/Vop1cXcf23AKgKl1ESlVGulVDul1H/j1TeNJlFkhDnQx8qY7iaLaolSbJ880B+bYVecZ69pIwknRkApRc+ePencuTMHDx5k4cKFKKX8KwgriouNtBdXXAHff2+sHLZsgb59QzysDbGudX26oXMraTRxxG2q61gb0wMN9XYEz/qdVgfB7SWL+GfdwUbpw4cPM2jQILKysgDYsGEDV11l754K8N13cPHFRsI8gJUr4Y47iKqWs9sVmI6QtkYLB40mjrgZpDPSU9k+eaA/r1KrsQu5cPLSUoPu/KxsLpy81PKY071XjO3Hc0O7u5r1h1odZPbI8J/js1sEey1NmDCBBg0asHDhQsaPH09RURFNmjSx7WNRkSEUOnY0BMSbbxr2hj59Qj5eSNwIZh0hbY8ot1XAKyC9e/dWq1evLu9uaDSusEtxPWlQF4CwjnmShFo1qtmWE43USyfUdXY1HJrVro53/p/x/T+uXr2aXr16Wbbpu3dWlhHMtmaN0caPP0Lr1pG+XWvmZ2Uz/v1N/j6nepKo4UmOaxnWyoCIrFFK9XY8RwsHjSZx2A2UdmU4fSuOUEWBfIIks0eGoxAKZyC0aseKoqMH+On9ZyjI/haAvXv30rRpU9s2UlR1vn/qcgDq1zeC2QYPdt2tsIjVu6hquBEOOiurRpNA7Fxgo80RFeh1Y5WlNdyYBHDn7XNk2asc+/pdAGbNmsXNN9/s2EbBvrpkv9fTv7127amMqvEgFvEZpytaOGg0FYBQKTnclBP16f/tAtzC9cpxOr+kMJ8jS6dzYv1iACZMf4ebby47/fe1Uaxs5YgAACAASURBVJJfjd1TjUTLybXyaHD1Ov71l0acdVZ8B2hdwyFytEFao6kAOBmD3Xo8+TyI7AjXK8fu/JItn7L72SGcWL+YjEuGMfer7Tx8h7VeqFl6Kvl76rF35sUAJNUopNkdn1OrU3ZC4gsSkX68qqJXDhpNBcBNSg7fsbqpHk4WFvkzqIIhSJwEQyReOX3bN2LWyl3+QD6lFPnrPuDgkn8C8Nxzz3HffffZXn/gAHzxkJG0L7luLo0GrSKtzUH/8UTM3ssjMWNVQQsHjaaC4JSSI/iYlWHbV4shmGSRiIzR89Zk+wVD4U+7+HnRcxTu+56ePXvy5ptv0qFDB9vrP/kEfvUr43O1WgU0/e3nJFUvLbwSMXvXNRwiRwsHjaYSYidIgmfJgpFLyafCcTsoBhpy83as4+DsRwBodc09rH5vKiJiKaB6Ncxg1CiYN89oZ9IkaN//Jx56FwIDsBM5e9c1HCJD2xw0mipESrXS/9K+mb/bFNu+YLvsnDxKvPn8vOQfHJz9KFKtOg2ufhDVsb9fMASn8L7nyf20aGEIhma/3EmLB/7L+ywFKJVWvZ5Zc3vU7HWuA/o0iUevHDRVglgVokk0seq3m5iEUC6cgW3k79rAgbf+DECtbgOod9kdJHlq+FVBgSsL7+E0Dn/UmfwdjUiqXkSjflvwdNsBlK5+t2JsP9vqeKAzo1Y0tHDQVHrKc8CJZnCPZb/dxCSAsxHY18aJTcv4+QOjBldah1/SYMBIoLQqyNfO4aUdOL7qbADq/uJ76l6wFalWUqrdQKHkVGdiyuItlUaonw5o4aCp9JRXoFO0g3ss++3W88fJCLxrTzZ7/n4bACnNO9Lw6gepVtfIixRcR7phch2+nXcOuVuMSOgGA9dRq7O9esjXP6d+Br+/yroarCpo4aCp9MQ60Mk3KGXn5PmrpgUPjhD94B7LftsF0QXiZARevXo1+/5lVNZJad6RJsMmIUlGbEVgHWml4O67YfVLF0NSCemXbKbOeduQJEWqJ5kaniTr3EumUArVz8BIb61+Kl+0QVpT6YlloFOgoRWwzT4K0Q/user3/KxscguLyuz3JAvpqR7HdODFxcV069aNPn36UK9eOs1v+gtn3PKUXzAECpTsbLj2WnjpJePaidN/puOAvSQlKX/7j1/TyTGzq5uAvr05eY6CV5MY9MpBU+mJZaCTk+4+eFUQKuWFDzv1SCz6bWeITk/1MO7aTo6z7P379/sT5AFs2bSRz3fmlenrtd0yuOACo8aCxwPPPgv33APJyY34M/0s27ZTBwXGHditIJqlp+q0FxUALRw0lZ5YBjqFGnwCj7sZ3N3YJaLpt50wq5lSzbGdRYsWMXz4cAB69erFqlWrEBEy65dW2/zzg/2kX5/D8Z3pADwx/SD3397YsU+h4gp8x+0ypjoF9Om0F4lDCwdNlSBWgU6hdOKBg5PblBdOdolo+x3uDLugoIAaNWoA0KVLFz799FM6duxY5ryiImjXpZBtW5ogniLqXvADdS/6npk/JtMuK3bV6sD+/em0F+WLFg4aTQBWqwEfVoNT4Cx4yuItjApyyYy3esROmKWnecrs27JlCzfddJN/e+XKlaSlpZU57/vv4bbbYNvm6sY9fvc51erkA7H3ArMTjjrtRfmjhYNGE0CwTtzJW8mHk+rIrV0iUkb3b8fod9aXSsIHcCK/iPlZ2WT2yEApxcUXX8yKFSto0KABCxYs4JprrinTVmEhtG0LO3dCejrUu/Q7ap+3rUwd50Tp/XXai/JFV4LTaKLEroqbzyjsVInMjS9/qHO6j19CTl5Z99GM9FQW3tmTzMxMPvvsMwC2b99Oy5Yty5y7aRN07nxqe+9eGPKa9XMli/DMjd30wF2JKddKcCJyJvA6cAZQAkxTSk0VkfrAbKAlsAO4USl1REQEmApcBeQCw5VSa+PVP030zM/KZtyCTf6BqV6ah8evcfaQidV9y1vdENgHu+mV771MGtTFsr9ujNWPzN9YKm124DlgrHCsBAPAls/fo/vLv2XPnj3cfPPNzJw5E4+ntLopNxcefxymTjW2r7/eyI0kYq9iK1ZKxxycBsRt5SAiTYGmSqm1IlIbWANkAsOBw0qpySIyFqinlBojIlcB92AIhz7AVKVUH6d76JVD+TE/K5vRc9fjLSn99+NJFoaee6arYvaR3re8awK7ra0MpQPIgnGqG+3LQzRq9jpL4ZOe6qGgqMSyD6qkmJzPXvOX7/zyyy85//zzy5y3di306mV8HjbMEBCNGpU+Z35WtmXZ0VDPpqnYRLVyEJH3wXZShFLqWqeGlVL7gH3m5+Mi8h2QAVwHXGqe9hrwKTDG3P+6MqTVShFJF5GmZjuaCsaUxVvKCAYAb7GynenGYvCuCDWB3eYxAnv9/PysbFuvqOycPFqNXUiSSMhVSTCFB7dx+JPpFOzaSL2GjVm/djVnnnlmqXOOHoX/+z945RVje/hwmDHDXA29UnaFM2r2urCeTVM1cFIrPR2rm4hIS6AH8BXQxDfgK6X2iYjPaToD2B1w2R5zXynhICIjgBEALVq0iFUXNWHiNDAED2ixHLwrQnBUOPeyMjz7Vh5OKLCtBW3Hzx++yIn1HwJwz/hnmfrofUiQNfmVV+COO4zPw4fD3/4G9eqVr1FdUzGxFQ5Kqc9icQMRqQXMA+5XSh0L/mMNPNWqGxb9mgZMA0OtFIs+asLHTS6fQOLtupnIgcrtswtY+uWHs/Kwazc9zePPYVTiLeDIslf9guGDDz5g4MCBpa7JzYVHHjGimwFGjTIEg1OffEI9lhHoFcFepHFHyNxKItJGRN4RkW9FZJvvx03jIuLBEAyzlFLvmrsPmPYIn13CV1R2DxC4/m0O7HX7IJrEMrp/OzxJtoK+DLF03XTK3ZMI3OQHAmNmYzXwRSMoBbjl/Bb+HEYnv/sfu/82mBNZC6l//iBmf/ljGcHw4otQs6YhGP74R0OtFCgYnPq0NyePzB4ZpYr12OVpCoVVgSA3BYg05YMbb6UZwOPAs0Bf4DdYz/JLYXofvQJ8p5QK/FNcANwOTDZ/vxewf6SIvI1hkD6q7Q0VF9/AEOytNLBrU+atyY5bZGu8gqPCmdEG9yHJjIUIJsMhsV44q65kEUqUKtUvpRQfzn2dfy54EoB2t4xj8oN3lOrzsWMwZgy8/LKx/cYbcOut1vcItSKLRcxBRbAXadwT0lvJtGr3EpGNSqku5r7/KaUuDnHdRcD/gI0YrqwAf8awO8wBWgC7gBuUUodNYfIiMADDlfU3SilHVyTtrVQxqWyqg2g9oOyuH9wrw9JrKxxvJ0+SMOWG0jEFP/74I23btkUpRf/+/fn73/9O69atS133wAOlVUhPPGGsHuL1DtzQauxCSwO7ANsnD7Q4ookXsYpzyBeRJOAHERkJZAPOmbcApdRy7FcYl1mcr4C7XfRHU8GpbJGt0c5orVYzfds3KrWCsvLaGv/+JsvaB4HUqlE6gd6SJUvo378/AA899BATJkwgKemUdvjIEbj3XnjzTWP77bdh6NCQj5CQdBUVwV6kcY+blcO5wHdAOvAEUBd4Sim1Mv7dc0avHDSxwG5GC4ZqKJLBMlQMg4/AwkJW+GbVXq+XJ554gokTJ1JSUsI///lPRowYUercP/wBZs+GEydg0CCYPh3q1nXV3ZDEYjVYEWJUNAYxWTkopVaZH09g2Bs0miqF3YxWwL8/3HiNWLncNktPZdu2bX610fDhw3nhhReoVauW/5xDh+Cuu+Cdd4ztNWugZ8+wbuNIrGpd62R6lYuQwkFElmHtUqpDIzUVHjczXitXTSG6eA03KpRQtodUTzK/SPqBnj0HATB48GBmzJjhP64U3HgjvGe6dIwYYdgZLBKtRkUsDcmVTeV4OuPG5vCngM81gMFA2ZqEmipFJGqEimaIdjvjtZrROkUv+7KdBt4n+LndxAY4xTs0rlbAqgmD2Qz06dOHf//735x99tn+4/v2QbNmp87/5hvo1MnliwmTihB4qEk8IeMclFJrAn5WKKUewHA11VRRIvFHr4g+7OHUIc7skcGKsf3YPnkgK8b2s3VDBUo9l91zAyFjA+wG18IDW1k1cTAAkpTMn55/2y8YlDJqOPvq8zRpAvn58RMMYG8wThLRMQpVGDdqpfoBm0lAL4xMq5oqSiRqhHCuSdQKw2nGG6oPTkV/Ap/L6blXjO3n+FzBKxSlSji8+B+c+OZjkmvVp855g6lz7nU8+v5mqnk89KifgS/b9kUXGakw2raN//vU2VlPT9yoldZgqF8FQ520HfhdPDulKV8iUSO4vSYa42a4g6BTlbRQffD9vj9E0rloVC6Bg27xyRz2vGhEqHkan02ToU+QnGa4GuUWFHPfQ/kcXmFc16EDfPopJCfHzljshK8dq+ysOoit6hJSrQR0UEqdrZRqpZRqo5S6AlgV8ipNpcVOjeDkj+72mnBUPYFEorayS7WhFK76kNkjwzHK2er5go874UtLkbLzS/bNuAeSq1GjZQ+aDn/OLxjydzZg15SB7FrcmvPPh+3b4dtvDcEAkb/PcMnskUGJjdu7tj1UTdwIhy8s9n0Z645oKg6R5C9ye02kM+1IBkG7nEBHbdJdW/Uh1HNFk+upsLCQ63s25/u3J1K9JJ9mtz1Lk6FPIJKEKoFjq1px4G2jDkNKxhG+77mQS19eSPfxS/xC0e69hZOewy12Ak9hxHVo+0PVwqmewxkYKbNTRaQHp6Kd6wAxdpbTVCQi8Ud3e02kUbKRChUr10m7oLMkEVqNXViq76GeyyraOaVa6DnX1q1bGTZsmH97165dXPevDWTn5JG3rSEH5xo+H6mtD1D/8k1Uq3uqvzl5XkbPXQ84x2gEe1VFi5MdJh7qLE354mRz6I9Rta058AynhMMxjBxJmipMJP7owQOpb1YfytDrZqYdy9QLTgZWKDvQuXkX+d4S/+ecPK/jQHnllVfy4Ycfkp6ezpw5c7jhhhsAGNWvHX8cc5KfPm0LQN1f/EDdi77HKsu9t0QxZfEW+rZvxJsrd5U5riDmtoDA79fqu9D2h6qFUz2H14DXRGSwUmpeAvukqaS4MY5GGiUbqVBxMmI7ZVUNZ6Bz66l1/Phx7r77bj780Ki7sH79en/Bqtdeg+HDjXMbdDlA6i830qJ5Etk59vfNzslj3hp7VU48bAE+QWmXckTbH6oObryVeonIJ0qpHAARqQc8qJR6JL5d01Q23A6SsViVuBEqoYSV79pWYxdaXu92oHOj8nrppZe46667SEpKYsyYMYwfP56UlBQKCoyMqRMnGuc98QQ88kgToAlgn6MJjFTeTpld45nQTifRq/q4EQ5XKqX8aiSl1BERuQrQwkFTinhH0oYrVNwKq2gHOqfrS0pKGDRoEO+ZOS4+/fRTLr7YyHb/0kswZYrhgdS9O8yZA23alG5jdP92jH5nPd7i0vN0T5JY1vD2Ee9KbbGsDqepmLgRDskikqKUKgAQkVQgJb7d0lRGKtps0q2winags7v+dz3rMmDAAD766CPAMEKfffbZ5OXBY4/B02aV9g8+gIEDjYF5+GTrgTnQ4J2e6mHctZ1sdf/JIiEznVoJAcB1zIROolf1cZOy+/+AazEqwoGRmXWBUuqpOPctJDpld8WioqVkdps2G8oOln3bN7Is1GNH8PVtDizjtecmADBt2jTuuOMORITJk43keAcPQr9+MHMmnHmm+3cXeJ/0NA8n8otKrSDcvG+re3mShCKlsBoOrN6XpnLjJmV3SOFgNjQA+BWGx9IRoKlSqtwL82jhUPGIdSqHaNqLVFhFI+QKCgoYM2YMU6dOBeD999/n6quv5sQJqF371HkffwyXBZS8ciPI7LK4pnqSyPeWuH4/TnYMK3SltqpHrCrBAezHKPV5I0b6DO29pLEklimZo00NEanqI9IU1e+99x6ZmZkA3HvvvTz55JPUqFGDxYvhzjtPnXfgADQOqqXoRgVml8U131vCs0O7u37vkdSU0Jx+OAXBtQVuAoYBPwOzMVYafRPUN00lxG6mH8kKIBZ1BCIRVuEa1pVSvPrqq9xxxx0APPzww0yYMIGjRyHVHFfbtoX//c9ImGeFG3uN7f0JL6bBKSV5MNrIfPritHLYDPwPuEYp9SOAiIxKSK80lRK7mf7qnYdD1lO2orzqCLgt1DNl8RZ27dzJ0f/+jeM7v6Fr16688sor9O7dm3nz4L77Tl27di3UrGl/TzdGcadBPZx34hTpHIgbw7am6uIU5z8YQ520TESmi8hlnIqS1mjKYDfTf+ur3RElh4smqV00hMqX5BOCW79Zy56Xf8vxnd/QqO9wBj72GiPe9iICQ4ZAcqqXr782ajA4CQawzwMVHF1u9w9o907mZ2Vz4eSltBq70J//KPhe6akePMmlW071JPPMjd20YDiNcYqQ/g/wHxGpCWQCo4AmIvIS8B+l1JIE9VFTSbCbvQZHH4c630eo2bSTqioaQ3YoW8WTizaRvfifHF+9APGkkP7L20nrfS3/eqOYQ+9dYLaiqDF4GdnVOnEukavAgp/jF63r88XWw6Wik+1UP26DAK3uo91SNa68lfwnG4V/bgCGVoQa0tpbqWJh5wWTbJGeAty5SDrZMOw8ioC4udSuXbuWXr16AZDW/mIaDBhJ0bEm7Hv1EgCqn5FD/V9tIiUjp9QzRlp21eo5BvfKcOVm6+QBNbp/Oy0MTmNi5soa4c1fBa4GDiqlOpv7xgG/Bw6Zp/1ZKbXIPPYQRhGhYuBepdTiUPfQwqFi4TSYBdocfPujGaydBj6wTlkdib/+/Kxsxi3YRE6el9zvv+SnBU+hir3UOKsrjW6cSO53zfl5YVdQSaS120fDa7OQpFP/UwI8O7R7RMIqmjiN0f3bMWr2Osv8R777V5R4FE3iiaUrayTMBF4EXg/a/6xS6unAHSLSEcMzqhPQDPhYRNoqpZwtZpoKhZM6pvdZ9WM6U42kjkG4huz5WdmMnrue/JPH2PO8kV67+hnn0Pja0ZDfmd1TfgFASsZhGlzxDZ7Gx8u00Sw9NWKvq2ir66V6ksgNyBbrQ7AvdqSFg8ZH3ISDUupzEWnp8vTrgLfNFB3bReRH4Dx0UaFKh53raCzjHyA8d8zAa8JhyuItnDywnYNzxwNQrW4TmtwyhZPfnM3hxV0BqNkxm663bKFfx0bMW5NraR8ZFaLUqFN/3dSdsBM+Vqm+AdvVhM6oqgkknisHO0aKyG3AaozsrkcwigqtDDhnj7mvDCIyAhgB+NMda04v5mdlc7KgKKxr3BqyfSilWPPiPRRkf0tSzXQaZf6ZpBrXcPCtdhTsrYekeOlw6zdMvLsxmT0MFY/d6sguB1IoYTW6fztGz11fJsFeYN0Jq6R8p57B3btx2x/N6UWihcNLwBMYk5cnMIoI/RZrF1nLP22l1DRgGhg2h/h0s2pRlTxR7FJI1Evz+BPTWeHTp7uJuv75559p2LCh/9qmw18gd0sPjszvBED9/huo1W03JwUeeveA/1q71ZHbxH5W+Z1COY97ixVJAlYJWu0cAeqlecj3luiMqhpHEioclFIHfJ9FZDrwgbm5Bzgz4NTmwN4Edq3KEm0KioqGXQqJtOrVHIVDoD3ESd/+6aefcuuttwJQp+EZVP/lfzg4uyPen+qQXDeXhlevo0bzI5bXgn0CvzxvsX+wTk/1UFhUzP2z13H/7HXUS/MwsGvTMoGCs1buslUBBVKirA3Mg3tlMHvV7lIrC0+y8Pg1nfzvoipMGDTxIaHCQUSaKqX2mZvXA9+YnxcA/xaRv2EYpNsAXyeyb1WVWKSgqEg4GWntVg/10jwhr88+fIImTZpw8OBB2rRpw1Ov/5dH/+8Cjr1bF4AGA9dRs1O2pR4/OyfP71kknFryZufklSrhWawUnmThWJ6XQDPxkVyvbalPt0wa1MUyBffsr3dbNhprG5Cm6hE34SAibwGXAg1FZA/wOHCpiHTH+BPdAfwBQCm1SUTmAN8CRcDd2lMpNpRXCop44ZTawqowjidZGNi1KRdOXmpbErTo6AGOLXqG4wcPAvDqqysZcFM1CvbXAaDp8P9Rvckx2z4Jp7ykQg3odvaBaEhP9VgO9hdOXlrGXuEtUYxbsEkLBk1I4umtNMxi9ysO508EJsarP6crFa0AT7T2Dyf9vZUrbd/2jUqpa4IFw6H//JXc778grVZtXn/930yaNIyLL4aktAIaXreGtHb7bb1+gFIrhXgQqn1PkjDu2k6Wx+wmADl5Xn8ajUipSnYsjTVOuZU0VYBQeYISic/+kZ2Th+KU/WN+VrbrNtzkIArkg/X7LG0U4i3g0PzJ5H7/BQD/mracp58exnffGcd73reKmu3LVzCkepK55fwWpZ711qDtKTfY5z9ymgCEymvlRCy+R03FJ24R0olAR0i7I56zvHDaDifiN9K+uMk2evLbz8hZPovinH0MHJjJBx/MAlJp2hRefhmuvdZ9W3YeQdGSLBJ14rv5WdncbxNjEU0Bn3h/j5r4U94R0poKQryMj+F6QjlFNftsAtEILztPJh9KKY59NY+cz2YC8PzznzBu3KnBbO1aOOMMSvXfJ/jshv9ipcp4CvlWFBnpqbRskFomUZ4nWaiWJORZRC9D7FJZZPbIKFV7OpBo1Iqh7Fha5VQ10GolTcQ4eUJZYTcg+Qy60aoonIzs3p93c3DuOHI+m0lSal1gM/fe24/atWHWLCNgbOW+0umtAVaM7cf2yQP9OZuC8am1Aj2i6qZ6eG5od1aM7ceOn8sKFm+xon7NFHZMHsiOyQO59fwWJJv6q2QRBveKnTB//JpOMVcrOqVS1yqnqoMWDpqICdcTysr+AWX19m5qPVhhN2idXPEme//1R/K3r+GMX4yhZvWfgHZ06AAbN8LNN4fWo4ey3eQHrAJy8rz+a93MsuetyfarpoqV4s2Vu+j46H9DDqhWtRqCCddG4wandxHuhMEKN8+liT9araSJmHA9oQJVNcExAcFEomoKdmVVxV5yPn+DY1+/a57xJvu/uIVzzoF/vwlXX33q2lDxIE5JBS+cvNT22lDvyE4VlustYfQ76wFrFV04Kr1YqxWd3kWkeaR8VLWgzcqMFg6aiHGbFiIQ30BlZ9T0ERg74HaAyOyR4U+vnbdtDQfnPg5AjbOvJ3/bG0BNunSBlSshLa30tW5WQXaDrNO1dum6fe/IURVWrGyDFe2E2YNz7AVKLLF7F9G6Tle1oM3KjFYraSImGpVFqJlkpKqmo3leTmxa5hcM1Zs8Sf62d4GaTJ0KGzaUFQzgrEd/ZP5GWj+0iJZjF9L6oUU8Mn+j62tDvaNQg2a4qrtipcpVxx+t63RVC9qszOiVgyYqIlVZRJJyO9QAsXfvXnY9M4iSokI8DXrg/fnfFB5oT90LfqDDVXu4996+ttfarYJaNkgtkwLDtz0hs4vjtb4B0ekdhSrK4yR47N5fpDPtWHgZhSqxGoqKFrR5OqNXDppywc447YTTALFy5UoyMjIoKSqkWu2b8f68CqQtDa5cT9PLtjJmYFvHtu1m+Cu3HbE8/62vdpe6dnCvjIg8jjJ7ZHDL+fap5/u2b2S5P9T7C1fwxtLLKLNHht/La8XYfqWSEoYyNFekoM3THb1y0JQLvgHjwTnrLYPIgo3VdgNEcXExzzzzDA8//LC5ZxJFx8fS4oqtSNctZDSowej+7lRdVjN8uyCywD5beRzNW5NN77Pqu7rvhMwuLNywzzIeYdnmQxZXnHp/dv1LdgrttiDeun63huZoVx6a2KGFg6bc8P3DWyXLG3rumSzbfMhxgNi3bx/NmjUDoGnTIezbNw2ox5w5cMMNrYHWYfcpWLVi51EVOPjGYmDNsUk37qRKy+yR4Up4uSHeuv5w3pHOGFsx0MKhklKlolCDxzFlVFXz6fStWLRoEbfffru5dRGHD8/hmWeE++6D5PC0VX6sZrfJSUKxRSWdYX1OlR+JRcRwpLr2DJvr7IL27Ii3rl8bmisfWjhUQhLtCx5PQTRl8RbLtNI+z6Tg+/ZrXZu6dY0aC7VrdwM+BzqwciV07x59X4Jnt8UliprVk8n3llCsFMkiDOtzZinB5TSwuv2uInELjua6eLVjhzY0Vz60cKiEJNIXPNaCKFjQ2BlOffcJvO8D0xax/eUR/nOKilby4os1+OMfIcmla4WToLObxeYWFjsmqevbvpFlsZ6+7Ru5/q4i1bXHSkcfb11/vIWPJvZo4VAJSeQSPZaCyErQOOn0fecppTiy9F+cWLcYpAGoPwFj+eor6GKveXJ1/0BBF+ns1s5o7LOZWBHL7ypWOvp46vq1obnyoYVDJSSRS/RYDm5WgkZh7ZnkO68k/wR7X7mb4hM/A02h+pc8OuYsxo1zv1oAQzBYeUYFCrpQs1u7VYdTtlm7lN7B31W8VYUVwUalDc2VCx3nUAlJpC+4UxBWuNgNor701oHxBcki5P7wFXtfuY/iE0eBi4BdtPj9j/zlL+ELhofe3WjrwePrl1M0s1McgNO7sLqn1XcVi4R1duhMqZpI0CuHSkgil+ix1BXbrXiCi8QUFxcz6LzWqKICACTlf6T/oj61z13sWJkNrGfIoeo8BA7udrNbp8Hb6h0FkyxCiVK235WbFVqks3+dr0gTCaedcKgIy+tYkKgleiwFUd/2jZi1cpdjcNuePXvIzLzVLxhgNxm//47kmtsBexfN+VnZZQrbBBu1rXCjNgLnwTs426wVJUo5GrVDqQqjUTtpN1JNJJxWwkGnA46MWAgiXxRxoGAQKJVm4tZbf8OsWTOBmsAManf6FfWvXu8/327F4lTSM89bbKv3TxYpozaK1FgdKttseprHMQV5qBVaNLN/7UaqiYTTSjjo5XVi8M3AAw2yVgO0wvDoycvLo9/Vd7Fy6UwA6nVczNS/t6J2XcWUxamlBlSgzCAbSm1kVcozK9Kj6gAAG9ZJREFUuBRnqL8Nt+o1q/M8ycKJ/CL/qsZqUhJqhRbN7N/Nik2jCSZuwkFEXgWuBg4qpTqb++oDs4GWwA7gRqXUERERYCpwFZALDFdKrY11n/TyOv4Ez8AD8w1ZseWzj0l76DJz615qdRtBnQE7+OsnG5g0qEspW4Td7N5JMIChivIJEbuZe6i/DTfqNZ9QDFytZKSncrKgiJy80ukx7GId7CYpkc7+3azYNBor4rlymAm8CLwesG8s8IlSarKIjDW3xwBXAm3Mnz7AS+bvmFLVl9fh2FPiZXsJNYv3oZRi/6w3KMyeA0CNs1+j0bVNSErZAVgPnnazezu1EZyaIYdSjbn523Bqw0oo+u4dbXU0iNwxwM592C42Q6PxETfhoJT6XERaBu2+DrjU/Pwa8CmGcLgOeF0ppYCVIpIuIk2VUvti2aeqHKUZjj0lFrYXK9WRXZ6fYLxHT3DwrVcoOvoRALW6fUCDAQBFpc4LHjydCtzYBdPleYu5f/Y6xr+/icev6WQ509+bk0fdVA+eZCmVANDt30aoGIpYTEoidQzQq2VNpCTa5tDEN+ArpfaJSGNzfwawO+C8Pea+MsJBREYAIwBatLDPg29FVY7SDMeeEq3txU51FKouNMCh+d+Qu2UsACnNH6bRkF+QnGJ9RfDg6ZRuIzCYrl6ah6O5XkoCjh/J9ZaqyRz8DDl5XjxJQr00Dzm5Xtd/G4/M31hGnx+Im1KhbonEMaCqr5Y18aOiGKStvNct/9+UUtOAaQC9e/cOLy8xVTdKM5wZYiSzycBZdpKDGscq4hmg+CQcmrOYgoMvAFC79zjqX9bb4kwDt8be4HsnCShFKcHgI7Ams5WA9JYo0qpXI+uxKyzbD2Z+VrajYIBTpUKhfCYlVXm1rIkviRYOB3zqIhFpChw09+8Bzgw4rzmwN8F9q9SEM0MMdzZpt1Kwwxfx7FM57Z17jPytwwDo0uU3HLnkapJrplheK2Y/rAZPd/EElDH+BuITgE4C0q09ZsriLY6CwW2p0HhSlVfLmviS6PQZCwBfEv7bgfcC9t8mBucDR2Ntb6hsuCmpGEg4KTXCTb/h1sjswxfxvPpPA6n30XG/YOjf/w42bHiVFhnpttcFl5YMxleCMtx6BT58AtBOECpg1Ox1rlJNOK20ksRo7f7Z62g5diE9/rKkXNJVVJWgT03iiZtwEJG3gC+BdiKyR0R+B0wGLheRH4DLzW2ARcA24EdgOnBXvPpVGYgkF45TXiArUqqd+urrpXkczw3HeJnqSeZPV7Tj5psP0rTpZaxZcxN16nTm4amzOX7pMFqNXUhuYRGeJClzXTiqjkjUIp5k8V/nVIM5eDVgl+OobqrH/mYK8rynlFs+m0ciBYTOqaSJhnh6Kw2zOXRZ8A7TS+nuePWlshGpwdiN6sIqmjjfa6WhP4WdGio4wC0jPZU7enVgysj9fPllTwBqdb2FVpk3M3s/eEuMNo7kevEkC6meJP8AWsPjbp4yPyubcQs2OaqOfATaPuqleUp5K7lRUQWSnZNHq7ELS82+nfI8hbJ5JAId9KmJhopikNYEEA/3w0DX02BCDRh2Rs3A1YZScN11Jfzhscfxep+CpJrU7ZNJ+i+HkVNQVjPvLVYUBbiNHsn1hnSnnZ+Vzei568tUjnNih0M+I58wbTV2oaPtwIdv9j1q9jrb2s2hiPQ7fGT+Rt76ardtNbpw7qXdWDVu0MKhAhJr90On3EM+QhWyX73zcKnBKTDCdvduaNHie8BQ2dRueT51r7mH5LS6jv2yU9/YCQerkqJOuHlf87OyHb2vrAh1ppgeU5H2KZhH5m8sVWmuWCn/tpOA0G6smmjQ9RwqILGu1+DGoOw0YPhSMASmwpi3Jpt312Tz4ovQrt0S4AIAOnbsRP0bHw4pGOxwElJu1D8+3LwvpzoPITKDO9+7WlIZmwqUtnmEw1tf7Q5rv49E1v3QVD20cKiAhGtcDkUoNUKoAcNKuBw7VJ3BvdO55x4hL68/bdqcwbp169i06Rsy6qVF1E9wdqd1GrCT5FQ6b1+J0SmLtzgaX+2EZrIIzw7tznNDu0ckJPK8JUy5oRvpAQbremkepgzpFtF3aLeqCbXaifXfkeb0QquVKiix9It3iixOT/Uw7tpOjvcKFC6qWDj29dnkfKGA8wHweDysW7eKtDRDKNhlJvUkCbkOxm8nITVuwSZHdU6JKnvfUGlB7IRmiVL+9NthR1lyKvAtVt+fU8rxUFTVoE9N/NErh3Ig3BiGaNvv276RrdtmQZGzpxKcms3nbm3MrqevIufz0VDUkeQaO3n99dcpLCz0C4bgzKRgzFinDOlGPZvAN985drPa+VnZIb2TMtJTwy61GaoEaiSG23iobYb1OTOs/RpNLNDCIcHE2/fcqv15a7IZ3CvDcqbppk7xA5e1I3fVORx6px2GNv4DAF5+9xN+/etfW94bSmcmzeyR4TjYOgVnheqf7x7heueE0smHMtwKcGHr+nFX20zI7MKt57fwf3/JItx6fouQ3koaTTRotVKCibfvuV37gd4uwTgN2vPmwZAhGcDXJFUfQEkhJHlq8Pbyb7nhvFau7n3/7HVMWbyFuqke2xWA0/M79S8wfsHOVddukA+VWsKqSE4gCtjxc16pmhPxYkJmFy0MNAlFC4cEE2/f80jasRo8Cwvhr3+F8eNLgN5AFs0aN+ett/7HRRddFPa9s3Py8CTb68idrrWzmdRL85RKkhdJkjk7nbxVkZxw+63RVGa0WinBhNJzx6t9J04WFJWyf6xeDb17w/jxioyM64AsAJrc+jS//uCopZ3EFy/ghLdYYeHhCRiz8JZjF9Jy7ELa/Hlhqfbt1D+PX9Op1L7MHhml1GfB8Rjh4DaflGMKDY2mEqNXDgkm3imUQ6W1tsKn6snOyWPMnG849K8zkJL/csklb9P+wgtZ9M05SPvL+EmJ/7xALyCneIFgShRl6jkH4y3BH4Hsm9k7BeH5sIvH6H1WfVdpRQLVS25jKkI5DOnEd5rKil45JJh4+54Hth8OBfvroEog35tLUtNbOHz4Gnbuy+KTvJYkdfgVEjQKBhqyw8na6nte3/M74WvfbtAPXr2MW7DJ0ubx4Jz1jp5hVkZ8t/ENObn2XlSnS+K7eHvfacoHvXIoB+Lte+5rf35WdsgcQCWFyeR83o7ja1pS94L3yd36IN6D27n33nv5uuGV7DsZOuWGW727VT3nlmMXhmw/lItqqOR5gZXqrOIe7Oosh6pqB85qvEQnviuPVUosSs5qKiZ65VCFyeyRUSpKNxARyN/ZgH2v/pLja1pRq8c2Tm6+i+LjP9NoyOOsbnqdo2CAUwNjepr1PdJTPVGtkELFG/gGonDSali57tq17ytaJBjPEmxQD6UOTGTiu/JapYQbW6KpPOiVQwUl2lmg7/qcPG+ZGbAnSTj8RWsOf9aO5LrZNL5pNalnHadg/2iSa9WnWq36IetB+wbG+VnZnMgvKnPckywhI68B2jSuyQ8HT1oe8xnKnRLjhWNb8RE8ONvZGHxFi3yE+50kMvFdeaXn1plfqy5aOFRAol2qB18fqCJpVieVvKIijjf/mbT2cyjIHkn+jktJPet2Us44p1Q7dqqVwJQbF05eapkptWb1aq6MwHuO5JfZL0C1JPEbysPJmOqG4MHZrZOAkzrQSnAksn5zeQ3SOvNr1UWrlSog0S7Vra4vyvOQu6Qn5x7ox5ET+eTv/Ae5W4YhycmktTnftq1A1UpGeirPDe3OuGs7MWXxFlqNXWir0jnqohiPnSE7SSSs1NzhkOpJpm/7RqUMqEBUTgJ2Kp1o2w2HeLtI26Ezv1Zd9MqhAhLtLDD4vNwtZ/DzR50oyatO0iW7OTz3UY7v2EDNjpdS/4q7SEqxz6JqpVoZ/c56vMXOg7ebQcnueWK9UvBRs3oyKqAWApwayCcN6uI60jl4lZBbWGQrzK3qYcfDcJzIVUogoaLMNZUXLRwqINEu1X3XF5+szuGPOpG7pRnVmxylw283cPvt1Xll5m6aXvsnqne41H+NJ1lAUWrGHjjL9v3jHzlZEFIwuB2UQpUfjYR0hxQduYXFljaUcHTzVio/O6yEX7y8e8pzkNaZX6smWjhUEAJnk3VNz5jAQTicWaBvFnmsoBr5OxpR98Is0uq/zbjf3Ev37hns27Obj77PKTOQAKXqMwuK2at2+/sRyitIIKxByW62O7hXBvPWZIdtbM4w723nvuskbtyuysKJ6QgU5tGUaXWLHqQ1sUQLhwpA8GwyJ8+LJ0mol+YhJ9cb1oCbnQ1blmTw1+vh6SVbKLnhFY4umczRgzs5e8ItQAY1a9Yks0dNS3VHYApvp9oLVmx3qNfspErx7U9P86AUzFq5i/Q0DynVkjiaZzx/3/aNWLb5EHtz8kj1JJXpm2+VE6kLpdtV2f+3d+5RVtXXHf/smbkDMzwcGB4hgDqAZiQpjjqxrkoV0CJaDJKaalxthpjWVVJXojUSolQlKRZDLT6aBugSY6rCqAmNlVhHAUk1hpe8xoKBVGyBEawFFbUEYfeP3+8O17n33MfcufecO7M/a911z+ve85095559fvv3++2d65wOyL9Mq2GEgTmHCJDqafTYCaW6suITieXSoQpLl8Itt7ikeQs+U0bbK//CWysXEavqy50/eIxx48a1H5/qZp3LU3FHguZTxM+VLpSSmIIjfsyhD49RFStn4TUNgYnxEvVPrB+ctrWRzbDcbAgKhdVUxejTqyLQ+eVTpjWqWGqQ7k0ozkFE9gDvA8eBj1W1UUQGAs3A6cAe4I9V9VAY+opNvh3Qa9fChAlu+eKL4Zqb32L2X/8p721fRe9R5zHoiptpbhtIw+Z9KW/E8Zt1Zx1DrMzNaQgimzH4Qcfc8sRWIDke3zGE0jC3JVD/8DTOI5tKeIlMrB+cMv351LOHBabUzrdMaxSxmdHdnzCHsk5U1QZVbfTrs4FVqnoGsMqv9wg6OwzxxAl44IGTjmHePFi9Gh7d8R9U1p3HgEl/zpCr76S8T03GXEiJldvSURUr508uOPUTwzOvOX9k+9DWVLl1gvoq9h3+qD0vT9Axx1UzzvTNVCnu5dmT+JurfidpWOl91zSw5c7JOd3M1ux8O6ftkP7/WKp1nW1mdPcnSmGlacAEv/wI8CLw7bDEFIKgZnhnhiG2tMDcufDLX0Lv3vDznx9j1aq5LFkygn2HR9LnrIuSPpMpF1K8ctsnaj+XCX17VwT2faR6gry5eQs3NW9p7yAOGn1UJmTVYsnUYZvuhiReY7ylke9NuDOtvKD/byk6hTg2M7r7E5ZzUKBFRBRYrKpLgKGq2gagqm0iMiTVB0XkBuAGgFNPPbVYerMmyAFk0wzPJn57/Dh873vOMQA8/DCcdtqLTJo0EYApX2qCUalrC8drD6RLFxHve8g2jhyUtC7xbwwalnpCs09/ke6mk26fkr7KXK50Zphxd5wLYDOjuz9hOYcLVXW/dwDPi8jObD/oHckSgMbGxsLMluok6RxAprh7Nk+1zz0Hd9wB69e79Y0bYd68L7JixQoAHnzwQZYdGQsBN8t41ChdSyXXp+tMT4rxcFW+E9vS3XQy1V/oyqfZzk42627DTMOadGcUj1Ccg6ru9+8HRWQFcD5wQESG+VbDMOBgGNryIZ0DyKcZfuwYnHkm7Nnj1pctg6lTP+CSSyax3nuKz/7Z33Hv3jqU4O+L1x7oyifZmuoYh9LUNIDU4aqqWDm9KspS9hV0HFmU6aaTqcBROseS64ib7tgK6Axmh+5P0Z2DiPQBylT1fb88Gfgu8DTQBMz37z8rtrZ8SecAOtsM37wZrr/+pGNobYXa2rcYP34K27ZtY/RZ4yj/wnc5ksW/MvFcXfEkG5SRtSNB4SpI7nOIT4KLz2nI5YY9919fS3JU6RxLZ0fcdLdWQGcxO3Rvwmg5DAVW+MpiFcDjqvpvIrIBeEJEvgb8F/ClELTlRToHkGsz/OhR6NfPtRpi/Y4yePp2xjQe5t6lL7H8h9+nqamJhQsXMmedZlXPIFYuKc+Vz1j1Bc+9njFBXjbhqsSZ4SJuEtyna6oC5zikInG+RLZ/T1hprg2jFBAtUJKzYtDY2KgbN24MW0Y7qWbCJo5KyfbGtW4dfPWrsGOHWx/xjRaOH9lF29IbAfjdiZex8snHqK2tpW72yozVygZUx7jzyuSx/Kn0xsqFPpUV7TOT091c050711QamWxXCIL0C+lnextGqSMimxKmEaQkSkNZS55McdhMzfB33oFBg9zyyJFw1owtfDh0H4dWP8R7G1ync+WnxlD2B7dSW1sLZF+oJhUpZ2Yf1/Z+gExhlnzOnY2WQj/F24gbwwjG6jl0MVedM5yXZ0/ijfl/mDJdcxBr18LgwSfXW1vhw0F7OLRmabtjGHTlrQxruo+2d08WyMknn342neHpJjZ1ZS7/MMbNWy0CwwjGnEPItLVBRYWb5VxXB48/7vIkHTiwi483NPPe+p/S9+wpjPjGMvqMvRhI7ljubEGZfJPN5XPubLUU8im+K/UbRnfD+hxCpKUFLrvMLU+YAM88A9XVypw5c7j//vvpXzuEyktvgsGj2z/TlXH4bLKFQufCRJnOm+3IJbtZG0bXk02fg7UcQmDvXujf3zmG+nrXWlizBg4d2ktZWRl333031dXVrHtpLffd+EcFe7Lt+OQ8oDpGrOyT+ZW6OswShZKahmFkxloOITBnjkuS19QEixa53EgrV65k6tSpAMRiMY4cOUJlZWXRtRU6DXNQkr2ubp0YhhGMjVaKKLNmwYwZMGaMW1+wYAG33XYbAPfccw+zZs0KTVuhJzZZwjbDKA3MOYRA//7utX//fmbOnElFRQXTp09n8eLFDBgwIGx5BaVUh49aYRujp2F9DiExc+ZMhg8fzgsvvMB1111Hc3Nzt3cMUJrDR4P6SdLVmDCMUsdaDiEwf/58Fi1aBMCmTZuor68PWVHxKMWEbZZmw+iJmHMIgfHjxzNlyhSWL1/OKaecEracolNqCdusn8ToiVhYKQTGjx/Ps88+2yMdQykSxgQ9wwgbcw6GkYFS7CcxjHyxsJJhZKAU+0kMI1/MORhGFpRaP4lh5IuFlQzDMIwkzDkYhmEYSZhzMAzDMJIw52AYhmEkYc7BMAzDSKKkU3aLyNvAm0U41SDgf4pwnq7AtBYG01oYTGvhSKf3NFUdHLAPKHHnUCxEZGOm3OdRwbQWBtNaGExr4chXr4WVDMMwjCTMORiGYRhJmHPIjiVhC8gB01oYTGthMK2FIy+91udgGIZhJGEtB8MwDCMJcw6GYRhGEuYc0iAiU0TkdRHZLSKzw9bTERHZIyLbRWSLiGz02waKyPMissu/h1aYWkSWishBEWlN2JZSnzge8LbeJiLnRkDrXSKyz9t3i4hckbDvO17r6yJyWRF1jhSRNSKyQ0ReE5Fv+u1RtWuQ3ijatreIrBeRrV7rXL+9TkTWeds2i0il397Lr+/2+0+PgNYficgbCXZt8Ntzvw5U1V4pXkA58BtgFFAJbAXGhq2rg8Y9wKAO274PzPbLs4F7QtR3EXAu0JpJH3AF8CwgwAXAughovQv4Vopjx/rroRdQ56+T8iLpHAac65f7Ab/2eqJq1yC9UbStAH39cgxY5232BHCt374ImOmXvw4s8svXAs1FtGuQ1h8BV6c4PufrwFoOwZwP7FbV/1TV3wLLgWkha8qGacAjfvkR4KqwhKjqL4D/7bA5SN804Mfq+BVQIyLDiqM0UGsQ04DlqnpUVd8AduOul4Kjqm2q+qpffh/YAQwnunYN0htEmLZVVT3iV2P+pcAk4Cm/vaNt4zZ/CrhERCRkrUHkfB2YcwhmOPDfCet7SX9Rh4ECLSKySURu8NuGqmobuB8mMCQ0dakJ0hdVe9/om+FLE0J0kdDqwxjn4J4aI2/XDnohgrYVkXIR2QIcBJ7HtVwOq+rHKfS0a/X73wVqw9KqqnG7zvN2XSgivTpq9WS0qzmHYFI9AURt3O+FqnoucDnwlyJyUdiC8iCK9v4hMBpoANqAe/320LWKSF/gJ8BNqvpeukNTbCu6XVPojaRtVfW4qjYAI3AtlrPS6ImUVhH5HPAdoB74PDAQ+LY/PGet5hyC2QuMTFgfAewPSUtKVHW/fz8IrMBdzAfizUX/fjA8hSkJ0hc5e6vqAf8DPAH8EyfDG6FqFZEY7kb7mKr+1G+OrF1T6Y2qbeOo6mHgRVx8vkZE4iWVE/W0a/X7TyH70GSXkaB1ig/jqaoeBR4mD7uacwhmA3CGH6lQietwejpkTe2ISB8R6RdfBiYDrTiNTf6wJuBn4SgMJEjf08BX/KiKC4B342GSsOgQk52Osy84rdf60Sp1wBnA+iJpEuAhYIeq/n3CrkjaNUhvRG07WERq/HIVcCmuj2QNcLU/rKNt4za/Glitvvc3JK07Ex4QBNc3kmjX3K6DYvWul+IL18P/a1zc8faw9XTQNgo3qmMr8FpcHy7muQrY5d8HhqhxGS5kcAz35PK1IH24Zu8PvK23A40R0PrPXss2/+MalnD87V7r68DlRdQ5HhcO2AZs8a8rImzXIL1RtO04YLPX1Arc4bePwjmo3cCTQC+/vbdf3+33j4qA1tXerq3Ao5wc0ZTzdWDpMwzDMIwkLKxkGIZhJGHOwTAMw0jCnINhGIaRhDkHwzAMIwlzDoZhGEYS5hyMHo2IHPfZK1tF5EkRqc7juyaIyDN++QuSJpOviNSIyNc7cY67RORbndVoGNlizsHo6Xykqg2q+jngt8BfJO70k4Zy/p2o6tOqOj/NITW4rJ6GEUnMORjGSf4dGCMip4urP/CPwKvASBGZLCKviMirvoXRF9prfuwUkZeAL8a/SERmiMg/+OWhIrLC597fKiK/B8wHRvtWywJ/3K0issEnTZub8F23i6tt8ALwmaJZw+jRmHMwDNpz41yOmz0K7ib8Y1U9B/gAmANcqi7R4Ubgr0SkNy4v0JXA7wOfCvj6B4C1qno2rmbEa7iaC7/xrZZbRWQyLlXE+bhkdOeJyEUich4udcs5OOfz+S7+0w0jJRWZDzGMbk2VT3sMruXwEPBp4E11ee/BJV8bC7zs0/VXAq/gsl++oaq7AETkUeAGkpkEfAVcJk3gXUmu0DfZvzb79b44Z9EPWKGqH/pzRCa/l9G9Medg9HQ+Upf2uB3vAD5I3ITLl//lDsc10HUpmgX4W1Vd3OEcN3XhOQwjayysZBiZ+RVwoYiMARCRahE5E9gJ1InIaH/clwM+vwqY6T9bLiL9gfdxrYI4zwHXJ/RlDBeRIcAvgOkiUuWz8F7ZxX+bYaTEnINhZEBV3wZmAMtEZBvOWdSr6v/hwkgrfYf0mwFf8U1goohsBzYBn1XVd3BhqlYRWaCqLcDjwCv+uKeAfupKbDbjspn+BBf6MoyCY1lZDcMwjCSs5WAYhmEkYc7BMAzDSMKcg2EYhpGEOQfDMAwjCXMOhmEYRhLmHAzDMIwkzDkYhmEYSfw/2OhBuBCA/RkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(preds_linear,Y_test) # scatter plot of predicted values against actual values\n",
    "# use numpy polyfit and poly1d to create a function for the line of best fit, then plot this function in blue\n",
    "p1 = np.poly1d(np.polyfit(preds_linear, Y_test, 1))\n",
    "plt.plot(Y_test,p1(Y_test),'b--')\n",
    "plt.title(\"Linear Regression, Rsq=%f\" % rsquared_linear) # add a title including the correlation coefficient\n",
    "plt.xlabel(\"Predicted\") # xlabel\n",
    "plt.ylabel(\"Actual\") # ylabel\n",
    "plt.plot(Y_test,Y_test,'k--') # add x=y line in black for comparison\n",
    "plt.show() # display the final plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `sklearn` function `mean_squared_error` to calculate the root mean squared error for this data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean squared error is 3121.49\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "print('Mean squared error is',np.round(mean_squared_error(Y_test,preds_linear),2)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysis of Coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can investigate the model coefficients to get an idea of the importance of each independent variable in predicting `Y`. However, we cannot directly compare coefficients as their dimensions are different. For example, the `AGE` coefficient will be inversely proportional to years, while the `BP` coefficient will be inversely proportional to mm of Hg. Hence to compare the coefficients we need to make them dimensionless and we also need to take into account the variability of each of the dependent variables. The variability of the features can be obtained from the standard deviation, which also has the same dimensions as the original variables.\n",
    "\n",
    "To compare the importance of the coefficients it is easiest to normalise the independent variables from the start. The dependent variable does not need to be normalised. We normalise all the variables such that their mean is zero and their standard deviation is one, which can be done by subtracting the mean and then dividing by the standard deviation. This is equivalent to multiplying the coefficients by the standard deviation.\n",
    "\n",
    "Normalising the coefficients is particularly important for regularization methods such as `Ridge` and `Lasso`, so that coefficients are treated uniformly by the penalty schemes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>SEX</th>\n",
       "      <th>BMI</th>\n",
       "      <th>BP</th>\n",
       "      <th>S1</th>\n",
       "      <th>S2</th>\n",
       "      <th>S3</th>\n",
       "      <th>S4</th>\n",
       "      <th>S5</th>\n",
       "      <th>S6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "      <td>4.420000e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>-3.717489e-17</td>\n",
       "      <td>1.321216e-16</td>\n",
       "      <td>-4.609686e-15</td>\n",
       "      <td>-9.625282e-16</td>\n",
       "      <td>-3.174936e-16</td>\n",
       "      <td>8.386956e-16</td>\n",
       "      <td>-1.446806e-16</td>\n",
       "      <td>-1.708035e-16</td>\n",
       "      <td>1.904051e-15</td>\n",
       "      <td>2.364248e-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>-2.251738e+00</td>\n",
       "      <td>-9.374744e-01</td>\n",
       "      <td>-1.895781e+00</td>\n",
       "      <td>-2.360375e+00</td>\n",
       "      <td>-2.662394e+00</td>\n",
       "      <td>-2.427874e+00</td>\n",
       "      <td>-2.148448e+00</td>\n",
       "      <td>-1.604285e+00</td>\n",
       "      <td>-2.648040e+00</td>\n",
       "      <td>-2.893112e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>-7.832846e-01</td>\n",
       "      <td>-9.374744e-01</td>\n",
       "      <td>-7.188104e-01</td>\n",
       "      <td>-7.697777e-01</td>\n",
       "      <td>-7.192046e-01</td>\n",
       "      <td>-6.375263e-01</td>\n",
       "      <td>-7.374604e-01</td>\n",
       "      <td>-8.293610e-01</td>\n",
       "      <td>-6.981574e-01</td>\n",
       "      <td>-6.967595e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>1.130443e-01</td>\n",
       "      <td>-9.374744e-01</td>\n",
       "      <td>-1.529591e-01</td>\n",
       "      <td>-1.190789e-01</td>\n",
       "      <td>-9.073818e-02</td>\n",
       "      <td>-8.020037e-02</td>\n",
       "      <td>-1.382738e-01</td>\n",
       "      <td>-5.443750e-02</td>\n",
       "      <td>-4.089059e-02</td>\n",
       "      <td>-2.263165e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>7.995940e-01</td>\n",
       "      <td>1.064282e+00</td>\n",
       "      <td>6.562083e-01</td>\n",
       "      <td>7.485196e-01</td>\n",
       "      <td>5.955183e-01</td>\n",
       "      <td>6.267323e-01</td>\n",
       "      <td>6.155415e-01</td>\n",
       "      <td>7.204860e-01</td>\n",
       "      <td>6.810788e-01</td>\n",
       "      <td>5.862581e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>2.325260e+00</td>\n",
       "      <td>1.064282e+00</td>\n",
       "      <td>3.581660e+00</td>\n",
       "      <td>2.772916e+00</td>\n",
       "      <td>3.232188e+00</td>\n",
       "      <td>4.174548e+00</td>\n",
       "      <td>3.804760e+00</td>\n",
       "      <td>3.889923e+00</td>\n",
       "      <td>2.805543e+00</td>\n",
       "      <td>2.847848e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                AGE           SEX           BMI            BP            S1  \\\n",
       "count  4.420000e+02  4.420000e+02  4.420000e+02  4.420000e+02  4.420000e+02   \n",
       "mean  -3.717489e-17  1.321216e-16 -4.609686e-15 -9.625282e-16 -3.174936e-16   \n",
       "std    1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00   \n",
       "min   -2.251738e+00 -9.374744e-01 -1.895781e+00 -2.360375e+00 -2.662394e+00   \n",
       "25%   -7.832846e-01 -9.374744e-01 -7.188104e-01 -7.697777e-01 -7.192046e-01   \n",
       "50%    1.130443e-01 -9.374744e-01 -1.529591e-01 -1.190789e-01 -9.073818e-02   \n",
       "75%    7.995940e-01  1.064282e+00  6.562083e-01  7.485196e-01  5.955183e-01   \n",
       "max    2.325260e+00  1.064282e+00  3.581660e+00  2.772916e+00  3.232188e+00   \n",
       "\n",
       "                 S2            S3            S4            S5            S6  \n",
       "count  4.420000e+02  4.420000e+02  4.420000e+02  4.420000e+02  4.420000e+02  \n",
       "mean   8.386956e-16 -1.446806e-16 -1.708035e-16  1.904051e-15  2.364248e-16  \n",
       "std    1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  \n",
       "min   -2.427874e+00 -2.148448e+00 -1.604285e+00 -2.648040e+00 -2.893112e+00  \n",
       "25%   -6.375263e-01 -7.374604e-01 -8.293610e-01 -6.981574e-01 -6.967595e-01  \n",
       "50%   -8.020037e-02 -1.382738e-01 -5.443750e-02 -4.089059e-02 -2.263165e-02  \n",
       "75%    6.267323e-01  6.155415e-01  7.204860e-01  6.810788e-01  5.862581e-01  \n",
       "max    4.174548e+00  3.804760e+00  3.889923e+00  2.805543e+00  2.847848e+00  "
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nX =(X-X.mean())/X.std() # create nX, a normalised version of X\n",
    "nX.describe() # show the descriptive statistics of nX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We repeat the linear regression, but now using the normalized variables. Again the square of the correlation coefficient is around 0.45."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training score is 0.4689\n",
      "Testing score is 0.4351\n"
     ]
    }
   ],
   "source": [
    "# split into a training set with 80% of the data, and a testing set as the remainder\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(nX,Y,test_size=0.8, random_state=42) \n",
    "linear = LinearRegression() # instantatiate the linear regression model\n",
    "linear.fit(X_train,Y_train) # fit the data to the model\n",
    "training_score = linear.score(X_train,Y_train) # calculate rsq for the training set\n",
    "# use the independent variables for the testing set to predict the target variable\n",
    "preds_linear = linear.predict(X_test) \n",
    "# calculate the correlation of the predicted and actual target variables\n",
    "rsquared_linear = r2_score(Y_test,preds_linear) \n",
    "# print the training and testing scores\n",
    "print(\"Training score is\",round(training_score,4))\n",
    "print(\"Testing score is\",round(rsquared_linear,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can plot the linear regression coefficients. It can now be seen that `AGE`, `SEX`, `S3`, `S4` and `S6` appear to have the weakest effect on the prediction of Diabetes onset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdQAAAFKCAYAAACkS65fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfZyVdZ3/8ddHICjRvAHUvAkyc01uRgXTVDJxFbNMM0srwSzxJkrLn6m1bXS3kTeZrJppmba1iiWWq2VZsVKmJrpoKK6GkkGKo64brEncfH5/nDN4GGdghvkO5xx8PR+Pecx1+70+58yZ8z7XdX2v60RmIkmSemaTehcgSdLGwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1XqQEQcEBH/Xe86NgYRsVNELI2IPvWuRepNBqpe0SJiQUQc3H56Zv4mM3etR03tRcSUiFheDaXnI+J3EbFvvevqqsx8IjMHZubKetci9SYDVWogEdG3k1nTM3MgMAiYCfxwA29f0joYqFIHIuLAiFhYM74gIv5fRDwQEf8bEdMjYkDN/HdGxJyaPciRNfPOiYj5EbEkIh6KiKNq5p0QEXdExEUR8RwwZW11ZeYK4AfA9hExuIvb3zMi/qu6/R9Wa/9y7eOMiLMj4ingu11o7+yIWFRt778jYlx1+t4RMTsi/hoRiyPi69XpQyMi28I6Il4XETdFxHMR8ceIOKmm7SkRcX1EfK/a/oMRMbrLfzipjgxUqeveB4wHhgEjgROgEljAVcDJwNbAt4CbIqJ/db35wAHAa4EvAN+PiO1q2n0L8BgwBPjK2gqIiFcBE4Bngf9Z1/ary98IXA1sBVwLHNWu2W2r814PTFpHe7sCk4ExmbkZcCiwoNrOxcDFmbk5sDNwfScP41pgIfA64L3Av7SFctURwHXAFsBNwCVre06kRmGgSl03LTP/kpnPAf8BtFSnnwR8KzPvzsyVmXkNsAzYByAzf1hdb1VmTgceBfauafcvmfmvmbkiM//WybbfFxHPA3+rbu+91b3VdW1/H6BvtfblmTkD+H27tlcBn8/MZdXtr629lUB/4M0R0S8zF2Tm/Go7y4E3RsSgzFyamXe1fxARsSOwP3B2Zr6YmXOAbwPH1yz228z8afWc678Bozp5TqSGYqBKXfdUzfALwMDq8OuBM6uHR5+vBt+OVPbAiIgJNYdPnweGUzkX2ubPXdj29Zm5BbANMBfYq2be2rb/OmBRrvktGO2315qZL3alvcz8I3AGlUPTT0fEdRHxuup6HwHeBDwcEfdExDs7eByvA57LzCU10/4EbF8z3v55HuC5XTUDA1XquT8DX8nMLWp+XpOZ10bE64ErqRwm3boainOBqFm/y1/5lJnPUDkUO6XmsHGn2weepHK+tXZ7O7ZvtquPp1rDv2fm/lSCN4GvVac/mpnHUTl0/TXgRxGxabu2/wJsFRGb1UzbCVjU1edAalQGqgT9ImJAzU9394auBE6JiLdExaYRcXg1NDalEjqtABHxYSp7qOstMx8Gfg58ugvbv5PKYdrJEdE3It7Nmoebu/V4ImLXiDioen74RSqHoFdWH9uHImJwZq4Cnq+2tcalMpn5Z+B3wFerz/VIKnu2P+jJcyI1AgNVgp9SCYa2nyndWTkzZ1M573gJlY5Cf6TaYSkzHwIupBJsi4ERwB0Faj6fSgeiIevY/t+B91AJreeBDwE3Uzkn2u3HQ+X86VTgGSqHZocAn6nOGw88GBFLqXRQOrbdoeQ2xwFDqeyt3kjl/O1t3Xz8UsMJv2BcemWJiLuByzPzu/WuRdqYuIcqbeQi4m0RsW31kO9EKpf83FrvuqSNjT3npI3frlSuCR1I5ZrY92bmk/UtSdr4eMhXkqQCPOQrSVIBBqokSQXU5RzqoEGDcujQofXYtCRJ6+3ee+99JjMHdzSvLoE6dOhQZs+eXY9NS5K03iLiT53N85CvJEkFGKiSJBVgoEqSVIA3dpDUVJYvX87ChQt58cWObhMslTFgwAB22GEH+vXr1+V1DFRJTWXhwoVsttlmDB06lDW/lU4qIzN59tlnWbhwIcOGDevyel0+5Fv9qqXfR8T9EfFgRHyhOn1YRNwdEY9GxPSIeNV61C9JXfLiiy+y9dZbG6bqNRHB1ltv3e2jIN05h7oMOCgzRwEtwPiI2IfKFwlflJm7UPmqp490qwJJ6ibDVL1tfV5jXQ7UrFhaHe1X/UngIOBH1enXAEd2uwpJaiIRwZlnnrl6/IILLmDKlCkbtIYTTjiBH/2o8tb70Y9+lIceeqjHbQ4cOLDD6U899RTHHnssO++8M29+85t5xzvewSOPPLJe25g2bRq77bYbH/zgB1m2bBkHH3wwLS0tTJ8+fZ2P46abbmLq1Knrtd3nn3+eyy67bL3W7apunUONiD7AvcAbgUupfHPF85m5orrIQmD7TtadBEwC2Gmnnda3Xklaw9Bzbina3oKph69zmf79+zNjxgzOPfdcBg0a1O1trFixgr59y3Vh+fa3v12srfYyk6OOOoqJEydy3XXXATBnzhwWL17Mm970pm63d9lll/Gzn/2MYcOGcdddd7F8+XLmzJkDwPvf//61rnvEEUdwxBFHdP9B8FKgnnbaaeu1fld067KZzFyZmS3ADsDewG4dLdbJuldk5ujMHD14cId3bZKkptC3b18mTZrERRdd9LJ5f/rTnxg3bhwjR45k3LhxPPHEE0Blj/JTn/oUb3/72zn77LOZMmUKEydO5JBDDmHo0KHMmDGDT3/604wYMYLx48ezfPlyAL74xS8yZswYhg8fzqRJk+joG8IOPPBAZs+ezcqVKznhhBMYPnw4I0aMWF3f/PnzGT9+PHvttRcHHHAADz/8MACPP/44++67L2PGjOFzn/tch4915syZ9OvXj1NOOWX1tJaWFg444AAyk7POOmv19qZPn756mfPPP58xY8YwcuRIPv/5zwNwyimn8Nhjj3HEEUfwta99jQ996EPMmTOHlpYW5s+fv/pxANx6663sueeejBo1inHjxgFw9dVXM3nyZABaW1s5+uijGTNmDGPGjOGOO+4AYMqUKZx44okceOCBvOENb2DatGkAnHPOOcyfP5+WlhbOOussnnzyScaOHUtLSwvDhw/nN7/5TZf+9muzXh+RMvP5iPhPYB9gi4joW91L3QH4S4+rUq8r/am+TVc+3Usbg4997GOMHDmST3/602tMnzx5MhMmTGDixIlcddVVfOITn+DHP/4xAI888gi//OUv6dOnD1OmTGH+/PnMnDmThx56iH333ZcbbriB8847j6OOOopbbrmFI488ksmTJ/PP//zPABx//PHcfPPNvOtd7+qwpjlz5rBo0SLmzp0LVPbKACZNmsTll1/OLrvswt13381pp53Gr3/9a04//XROPfVUJkyYwKWXXtphm3PnzmWvvfbqcN6MGTOYM2cO999/P8888wxjxoxh7Nix/OEPf+DRRx/l97//PZnJEUccwaxZs7j88su59dZbmTlzJoMGDeItb3kLF1xwATfffPMa7ba2tnLSSScxa9Yshg0bxnPPPfeybZ9++ul88pOfZP/99+eJJ57g0EMPZd68eQA8/PDDzJw5kyVLlrDrrrty6qmnMnXqVObOnbt6b/jCCy/k0EMP5bOf/SwrV67khRde6PAxdkeXAzUiBgPLq2H6auBgKh2SZgLvBa4DJgI/6XFVktTgNt98cyZMmMC0adN49atfvXr6nXfeyYwZM4BKANYG7jHHHEOfPn1Wjx922GH069ePESNGsHLlSsaPHw/AiBEjWLBgAVDZQzzvvPN44YUXeO6559h99907DdQ3vOENPPbYY3z84x/n8MMP55BDDmHp0qX87ne/45hjjlm93LJlywC44447uOGGG1bXevbZZ3frOfjtb3/LcccdR58+fdhmm21429vexj333MOsWbP4xS9+wR577AHA0qVLefTRRxk7dmyX2r3rrrsYO3bs6ktWttpqq5ct88tf/nKN861//etfWbJkCQCHH344/fv3p3///gwZMoTFixe/bP0xY8Zw4oknsnz5co488khaWlq69dg70p091O2Aa6rnUTcBrs/MmyPiIeC6iPgy8F/Ad3pclSQ1gTPOOIM999yTD3/4w50uU9tbdNNNN11jXv/+/QHYZJNN6Nev3+plN9lkE1asWMGLL77IaaedxuzZs9lxxx2ZMmXKWi/l2HLLLbn//vv5+c9/zqWXXsr111/PN77xDbbYYovVe2Zrq68ju+++++rOT+11dPi5bfq5557LySefvNa2O5OZ66xr1apV3HnnnWt8mGnT9rwC9OnThxUrVrxsmbFjxzJr1ixuueUWjj/+eM466ywmTJiwXvW26U4v3wcyc4/MHJmZwzPzi9Xpj2Xm3pn5xsw8JjOX9agiSWoSW221Fe973/v4znde2o9461vfurrzzg9+8AP233//9W6/LTwHDRrE0qVLOw22Ns888wyrVq3i6KOP5ktf+hL33Xcfm2++OcOGDeOHP/whUAmr+++/H4D99ttvjVo7ctBBB7Fs2TKuvPLK1dPuuecebr/9dsaOHcv06dNZuXIlra2tzJo1i7333ptDDz2Uq666iqVLKxeGLFq0iKeffrrLj3vffffl9ttv5/HHHwfo8JDvIYccwiWXXLJ6vLMPDG0222yz1XuwUDnXPWTIEE466SQ+8pGPcN9993W5vs54pyRJ6oEzzzxzjTf2adOmceKJJ3L++eczePBgvvvd765321tssQUnnXQSI0aMYOjQoYwZM2atyy9atIgPf/jDrFq1CoCvfvWrQCUsTz31VL785S+zfPlyjj32WEaNGsXFF1/MBz7wAS6++GKOPvroDtuMCG688UbOOOMMpk6dyoABAxg6dCjf+MY3GDt2LHfeeSejRo0iIjjvvPPYdttt2XbbbZk3bx777rsvULkc5/vf/z5Dhgzp0uMePHgwV1xxBe95z3tYtWoVQ4YM4bbbbltjmWnTpq0+j71ixQrGjh3L5Zdf3mmbW2+9Nfvttx/Dhw/nsMMOY/jw4Zx//vn069ePgQMH8r3vfa9Lta1NdLbL3ptGjx6dfh9qfdkpSc1q3rx57LZbRxcYSGV19FqLiHszc3RHy/ttM5IkFWCgSpJUgIEqSVIBBqqkplOPvh96ZVmf15iBKqmpDBgwgGeffdZQVa9p+z7UAQMGdGs9L5uR1FR22GEHFi5cSGtra71L0UZswIAB7LDDDt1ax0CV1FT69eu3+pZ0UiPxkK8kSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUQJcDNSJ2jIiZETEvIh6MiNOr06dExKKImFP9eUfvlStJUmPqzheMrwDOzMz7ImIz4N6IuK0676LMvKB8eZIkNYcuB2pmPgk8WR1eEhHzgO17qzCpzdBzbumVdhdMPbxX2pX0yrRe51AjYiiwB3B3ddLkiHggIq6KiC07WWdSRMyOiNmtra3rVawkSY2q24EaEQOBG4AzMvOvwDeBnYEWKnuwF3a0XmZekZmjM3P04MGDe1CyJEmNp1uBGhH9qITpDzJzBkBmLs7MlZm5CrgS2Lt8mZIkNbbu9PIN4DvAvMz8es307WoWOwqYW648SZKaQ3d6+e4HHA/8ISLmVKd9BjguIlqABBYAJxetUJKkJtCdXr6/BaKDWT8tV44kSc3JOyVJklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQV0OVAjYgdI2JmRMyLiAcj4vTq9K0i4raIeLT6e8veK1eSpMbUnT3UFcCZmbkbsA/wsYh4M3AO8KvM3AX4VXVckqRXlC4HamY+mZn3VYeXAPOA7YF3A9dUF7sGOLJ0kZIkNbr1OocaEUOBPYC7gW0y80mohC4wpJN1JkXE7IiY3draun7VSpLUoLodqBExELgBOCMz/9rV9TLziswcnZmjBw8e3N3NSpLU0LoVqBHRj0qY/iAzZ1QnL46I7arztwOeLluiJEmNrzu9fAP4DjAvM79eM+smYGJ1eCLwk3LlSZLUHPp2Y9n9gOOBP0TEnOq0zwBTgesj4iPAE8AxZUuUJKnxdTlQM/O3QHQye1yZciRJak7eKUmSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKkAA1WSpAIMVEmSCjBQJUkqwECVJKmALgdqRFwVEU9HxNyaaVMiYlFEzKn+vKN3ypQkqbF1Zw/1amB8B9MvysyW6s9Py5QlSVJz6XKgZuYs4LlerEWSpKZV4hzq5Ih4oHpIeMvOFoqISRExOyJmt7a2FtisJEmNo6eB+k1gZ6AFeBK4sLMFM/OKzBydmaMHDx7cw81KktRYehSombk4M1dm5irgSmDvMmVJktRcehSoEbFdzehRwNzOlpUkaWPWt6sLRsS1wIHAoIhYCHweODAiWoAEFgAn90KNkiQ1vC4HamYe18Hk7xSsRZKkpuWdkiRJKsBAlSSpAANVkqQCDFRJkgowUCVJKsBAlSSpgC5fNiNJ2vgMPeeWXml3wdTDe6XdRuYeqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBXQ5UCPiqoh4OiLm1kzbKiJui4hHq7+37J0yJUlqbN3ZQ70aGN9u2jnArzJzF+BX1XFJkl5xuhyomTkLeK7d5HcD11SHrwGOLFSXJElNpafnULfJzCcBqr+H9LwkSZKazwbrlBQRkyJidkTMbm1t3VCblSRpg+hpoC6OiO0Aqr+f7mzBzLwiM0dn5ujBgwf3cLOSJDWWngbqTcDE6vBE4Cc9bE+SpKbUnctmrgXuBHaNiIUR8RFgKvCPEfEo8I/VcUmSXnH6dnXBzDyuk1njCtUiSVLT8k5JkiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBfQt0UhELACWACuBFZk5ukS7kiQ1iyKBWvX2zHymYHuSJDUND/lKklRAqUBN4BcRcW9ETCrUpiRJTaPUId/9MvMvETEEuC0iHs7MWbULVIN2EsBOO+1UaLOSJDWGInuomfmX6u+ngRuBvTtY5orMHJ2ZowcPHlxis5IkNYweB2pEbBoRm7UNA4cAc3variRJzaTEId9tgBsjoq29f8/MWwu0K0lS0+hxoGbmY8CoArVIktS0vGxGkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCSt4cXxIw9Jxbire5YOrhxduUVJZ7qJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUgIEqSVIBBqokSQUYqJIkFWCgSpJUQJFAjYjxEfHfEfHHiDinRJuSJDWTHgdqRPQBLgUOA94MHBcRb+5pu5IkNZMSe6h7A3/MzMcy8+/AdcC7C7QrSVLT6Fugje2BP9eMLwTesrYVnn32Wa6++uoCm9b6Gv+qZ3ul3auvbi3eZjPVCr1Tb2/VKjXb/1cjKxGo0cG0fNlCEZOASQDbb799lxu/67Hyf+x93rB18Tahd2qF3qm3t56D3tBMtUJz1ev/V+/U20y1NtPrFRr7NRuZL8u+7jUQsS8wJTMPrY6fC5CZX+1sndGjR+fs2bO71P7Qc27pUX0dWTD18OJtQu/UCr1Xr+T/V+/U20y1Npt6v2Yj4t7MHN3RvBLnUO8BdomIYRHxKuBY4KYC7UqS1DR6fMg3M1dExGTg50Af4KrMfLDHlUmS1ERKnEMlM38K/LREW5IkNSPvlCRJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBRS59aAkSRtCI3/jjnuokiQVYKBKklSAgSpJUgEGqiRJBRiokiQVYKBKklSAgSpJUgEGqiRJBXhjB0kqrJFvPqDeY6BKr2C+8UvlNHyg+g8vSWoGnkOVJKkAA1WSpAJ6FKgRMSUiFkXEnOrPO0oVJklSMylxDvWizLygQDuSJDUtD/lKklRAiUCdHBEPRMRVEbFlgfYkSWo66wzUiPhlRMzt4OfdwDeBnYEW4EngwrW0MykiZkfE7NbW1mIPQJKkRrDOc6iZeXBXGoqIK4Gb19LOFcAVAKNHj86uFihJUjPoaS/f7WpGjwLm9qwcSZKaU097+Z4XES1AAguAk3tckSRJTahHgZqZx5cqRJKkZuZlM5IkFWCgSpJUgIEqSVIBBqokSQUYqJIkFdDwXzDeTPwydEl65XIPVZKkAgxUSZIKMFAlSSrAQJUkqQADVZKkAgxUSZIKMFAlSSrA61AlNQWv81ajcw9VkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCDFRJkgowUCVJKsBAlSSpAANVkqQCDFRJkgowUCVJKiAyc8NvNKIV+FMvND0IeKYX2u0N1to7rLX3NFO91to7rBVen5mDO5pRl0DtLRExOzNH17uOrrDW3mGtvaeZ6rXW3mGta+chX0mSCjBQJUkqYGML1CvqXUA3WGvvsNbe00z1WmvvsNa12KjOoUqSVC8b2x6qJEl1YaBKklSAgSpJUgF9613A+oiIf8jMh6vD/TNzWc28fTLzrvpVt6aI+FfgM5m5pN30fwAuycyD61NZxyLiU2ubn5lf31C1dEVEHAm8EfhDZv683vVow4qI1wCTgQT+FTgWeA/wMPDFzFxax/K6JCIeycw31bsO9Vyz7qH+e83wne3mXbYhC+mCp4A5EfEBqLwBRMR5wE3ApXWtrGObreOnYUTEZcAnga2BL0XE5+pc0nqJiIbqORkRfSLi5Ij4UkTs127eP9Wrrk5cDWwDDANuAUYDFwABfLN+ZXUsIpZExF+rP0siYgmwc9v0etdXKyJG1gz3i4h/ioibIuJfqh9kGkZETI6IQdXhN0bErIh4PiLujogRG6yOZuzlGxH/lZl7tB/uaLwRRMQw4BIqgfQ64Hrgy5n5Ql0La3IRMRcYlZkrq//gv8nMvepdV0ciYqvOZgH3Z+YOG7KetYmIbwOvAX4PHA/cnpmfqs67LzP3rGd9tSJiTma2REQATwLbZWZWx+/PzJHraGKDqh6xei1wVmYurk57PDOH1beyl6v9W0fEhVQ+uH4XOBLYOjMn1LO+WhHxYGbuXh2+Bfh2Zt4YEQcCX8nM/dbaQCFNeciXyuGdjoY7Gm8EbTX1pXJUYF6jhmlETFvb/Mz8xIaqpQv+npkrATLzheqbaKNqu391bY1ZHR9Sl4o6t3dbEEXEJcBlETEDOI41628Y1RD9aVb3EKrjDfdekJkfj4i9gGsj4sdUPmg3XJ1VtX/rccCYzFweEbOA++tUU2dqs2xIZt4IkJn/GREb7MhaswbqDtU3/qgZpjq+ff3KernqIbITgM9m5vSI2B64OCI+CpyamQ/VtcCXOwWYS2Uv+i806Bto1T9ExAPV4aBy6OyB6nA22N7JY8C4zHyi/YyI+HMd6lmbV7UNZOYKYFJEfB74NTCwblV1bHZEDMzMpZl5YtvEiNgZWLKW9eomM++NiIOpnPu9HRhQ55I689qIOIrKTkD/zFwODfth5UcRcTXwReDGiDgDmEHlg8DL/ud6S7MG6lk1w7PbzWs/Xm+DgT3aOiVl5iLgvRFxGHADsFs9i+vAdsAxwPuBFcB04IbM/J+6VtWxRnvu1uYbwJZ0/M993gauZV1mR8T4zLy1bUJmfiEiFtF45yW/RSXklwJExATgaCrP85F1rKtDETEG+HNmPgVMi4hNgS9ExMXAFzLzufpWuIbbgSOqw3dFxDaZuTgitqXBvnEmMz8bEScA1wI7A/2BScCPgQ9uqDqa8hzqxqJ9D+VGU92bPg74FHB2Zv5bnUtap2rHhGezwV7Y7d5Ia9/4/wRMaaQ30mqtCzPzyep4I9d6H3BwZj4XEWOB64CPAy3Abpn53roW2E4z1dtkr4P2/18TqdS6gA1Ya1P28o2I/at/3LbxH0XEr6s/B9WztvYi4vqa4a+1m/0fG7icLouIPYEzgA8BPwPurW9FLxcR+0TEf0bEjIjYo9pJaS6wOCLG17u+dr4F/B2g+kY6Ffge8L803v1RvwUsg6aotU/Nm+X7gSsy84bM/ByVy6kaTTPV20yvg/b/X18FrmED19qsh3y/QOVTXZtdqZyn3BT4DJVzPY1il5rhfwTOrhnv8Etq6ykivgC8E5hH5dPzudXzaI3oEip/79dS+Zsflpl3ReUa32uBW9e28gbW4RspcENEzKljXR1pqlojom/1NTqOymG+No34/tZM9TbV66ARam3KPVRg83adeR7NzHszcxYNdq0ka+/B11CHJas+RyWgRlH5lHdfRDwQEX+o6QDUKPpm5i8y84fAU2039Gi76UeD6RMRbW+Y41jzQ1/DvZE2Ua3XArdHxE+AvwG/gcq1iFT2ThpNM9XbTK+Dhqi10Z6UrtqidiQz31Mzus0GrmVdXhMRe1D58PLq6qFUqPREfXX9yupUw10Ptxaraob/1m5eo31YaXsjfYbGfyNtmloz8ysR8Ssqnel+UXPufBPWPIrVEJqs3qZ5HdAgtTZlp6SI+A/g8sy8pd30d1K5FOXw+lT2chExk5euN4Q13+gjM9++4avqngbu6LMS+D9e+nDSdpzz3kEAAAGYSURBVG1vAAMys1+9autIROzDS2+k/1ed9iZgYGbeV9fi2mmmWtV7mul10Ai1NmugvpHKbcZ+B7Q9UXsBbwXemZmP1Ku29iJibyq9z9p6ytWl91lXVV+UU4HngC8B/wYMovIJekLtpRSSpJc0ZaBC5ZITKtcX7U5lr+9B4FHguMz8WD1rq9VM3eQBImI2L3X0uYJ2HX2ywW7rKEmNomkDtU31/ORxwPuAx6nchOCS+lb1koi4PzNHVYcvBVozc0p1fE5mttSzvvZqa4qIeZm5W828hrtPsiQ1iqbslFQ9Ln4slSB9lsrdfBr1fGQzdZOH5uroI0kNoxHf0LviYSq9uN6VmX8EiIhP1rekTjVE77NuGBWVr5EKKr2S275SKmjce45KUt015SHf6g2bj6XSCelWKuclv50N+BVI0Bi9zyRJvaspA7VN9cbSR1I59HsQlVtN3ZiZv6hrYZKkV5ymDtRaUfkC52OA92dmQ93PV5K08dtoAlWSpHpq1nv5SpLUUAxUSZIKMFAlSSrAQJUkqQADVZKkAv4/7FIMsFqiJEcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "feature_names = X.columns.tolist() # write the column names to a list\n",
    "# create a new dataframe with the regression coefficients from the normalised data\n",
    "ncoefs = pd.DataFrame(linear.coef_.transpose(),columns=['Normalised'],index=feature_names) \n",
    "# add our original coefficient importance to this dataframe\n",
    "ncoefs.columns =['Normalised Coefficients'] \n",
    "# do a similar horizontal plot as before\n",
    "ax = ncoefs.plot(kind='bar',figsize=(10,5))\n",
    "plt.title('Linear Regression')\n",
    "plt.axhline(y=0, color='.5')\n",
    "plt.subplots_adjust(left=.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, we need to consider the variability of the coefficients as the training set is varied. This gives an indication of the variance or overfitting of the model.\n",
    "\n",
    "To investigate the variability we can use the `sklearn` methods `cross_validate` and `RepeatedKFold`. The first of these performs a number of runs of a model. The second splits the data in n sections and repeats the calculations m times. This gives n.m runs to investigate the variability of the coefficients. The variability of these can then be plotted using a box and whisker plot.\n",
    "\n",
    "We can see `AGE`, `SEX`, `BP`, `BMI` and `S6` have very low variance, whereas `S1`-`S5` have high variance indicative of overfitting. Of the low variance features, `AGE` and `S6` seem to have little effect on the predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing score is 0.4862 ± 0.0519\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZ/UlEQVR4nO3dfbRddX3n8fe3IUIUDBVCgQC5jFIIDWLtLeBIh/pQxMKAdNFKtApOWkZHmHZGhwSzZoC2TMNi1YdCnYoCAcsEEKVSQIvtRGtUtBfkOYhBI88aQCKMwbmE7/yx99WTm3NuzuWe393n3Lxfa53FOXvv89vfuzk5n7N/v/0QmYkkSSrjl5ouQJKkmcyglSSpIINWkqSCDFpJkgoyaCVJKsiglSSpIINWmoSI+K2I+E7TdcwEEbFfRDwbEbOarkUqyaCV2oiI9RHx5vHTM/OrmXlgEzWNFxHnRMRoHVZPR8TXI+J1TdfVrcx8MDN3zszNTdcilWTQSgMgInboMOvqzNwZ2B1YDXxmmtcvaRsMWmkSIuK3I+LhltfrI+KDEXFnRGyMiKsjYqeW+cdFxO0te5yvbpm3LCIeiIhnIuLeiDixZd6pEfG1iPhIRDwFnDNRXZn5PHAlMD8i5nW5/tdGxLfr9X+mrv0vWv/OiFgaEY8Dl3XR3tKIeKRu7zsR8aZ6+mERMRIRP4mIH0bEh+vpQxGRYyEeEXtHxPUR8VRErIuIP25p+5yIuCYirqjbvycihrv+Hyc1yKCVpu4PgGOA/YFXA6dCFWTApcB/BHYDPgFcHxE71u97APgtYC5wLvB3EbFXS7uHA98D9gDOm6iAiHgJ8G7gSeDH21p/vfx1wErgFcAq4MRxze5Zz1sAnLaN9g4ETgd+MzN3Ad4CrK/b+Rjwscx8OfBK4JoOf8Yq4GFgb+Ak4H+OhXXteOAqYFfgeuCiibaJ1C8MWmnq/jozH83Mp4B/AF5TT/9j4BOZ+c3M3JyZlwM/A44AyMzP1O97ITOvBr4LHNbS7qOZeWFmPp+Zmzqs+w8i4mlgU72+k+q9222t/whgh7r20cz8HPCtcW2/AJydmT+r1z9Re5uBHYGDI2J2Zq7PzAfqdkaBV0XE7pn5bGbeMv6PiIh9gSOBpZn5XGbeDnwKeFfLYmsy86Z6TPfTwKEdtonUVwxaaeoeb3n+U2Dn+vkC4AN1N+vTdSDuS7XHRkS8u6Ub9mlgEdVY65iHulj3NZm5K/ArwN3Ab7TMm2j9ewOP5JZ3FRm/vg2Z+Vw37WXmOuBPqbq4fxQRV0XE3vX7lgC/CtwXEf8aEce1+Tv2Bp7KzGdapv0AmN/yevx23smxYw0Cg1Yq5yHgvMzcteXx0sxcFRELgE9SdbfuVofl3UC0vL/rW2tl5hNUXbrntHQ/d1w/8BjVeG7r+vYd32y3f09dw//OzCOpAjmB8+vp383MxVRd4OcD10bEy8a1/SjwiojYpWXafsAj3W4DqV8ZtFJnsyNip5bHZPeePgm8NyIOj8rLIuLYOkxeRhVGGwAi4j1Ue7QvWmbeB/wjcGYX6/8GVXfv6RGxQ0ScwJbd1pP6eyLiwIh4Yz3+/BxVV/bm+m/7w4iYl5kvAE/XbW1xSk9mPgR8HfjLelu/mmpP+MqpbBOpHxi0Umc3UQXG2OOcybw5M0eoxjUvojpAaR31gVKZeS/wV1SB90PgEOBrPaj5AqoDl/bYxvr/H/B7VGH2NPCHwA1UY66T/nuoxmdXAE9QdfHuAXyonncMcE9EPEt1YNTJ47qkxywGhqj2bq+jGh/+0iT/fqnvhDd+lwQQEd8E/jYzL2u6FmkmcY9W2k5FxFERsWfddXwK1alJX2y6Lmmm8Yg9aft1INU5rTtTndN7UmY+1mxJ0sxj17EkSQXZdSxJUkEGrSRJBfXVGO3uu++eQ0NDTZchSdKk3HrrrU9k5rx28/oqaIeGhhgZGWm6DEmSJiUiftBpnl3HkiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQVZNBKklRQX12CUZLU3yJiUst7K9Ye7tFGxKyI+HZE3FC/3j8ivhkR342IqyPiJb1alySpGZm51WPB0hvaTjdkK73sOv4TYG3L6/OBj2TmAcCPgSU9XJckSQOhJ0EbEfsAxwKfql8H8Ebg2nqRy4G39WJdkiQNkl7t0X4UOBN4oX69G/B0Zj5fv34YmN+jdUmSNDCmHLQRcRzwo8y8tXVym0XbdtZHxGkRMRIRIxs2bJhqOZIk9ZVe7NG+Hjg+ItYDV1F1GX8U2DUixo5q3gd4tN2bM/PizBzOzOF589renF6SpIE15aDNzLMyc5/MHAJOBv5PZr4TWA2cVC92CvD5qa5LkqRBU/KCFUuB/xoR66jGbC8puC5JkvpS9NN5TsPDwzkyMtJ0GZIk4NBzb2bjptGetjl3zmzuOPvonrbZDyLi1swcbjfPK0NJktrauGmU9SuO7WmbQ8tu7Gl7g8BrHUuSVJBBK0lSQQatJEkFGbSSJBXkwVCSpLZ2WbiMQy5f1uM2obo0/vbDoJUktfXM2hUeddwDdh1LklSQQStJUkEGrSRJBRm0kiQVZNBKklSQQStJUkGe3iOpKxExqeX76c5gUpMMWkld6RScQ8tu7Pm5luofvT7vde6c2T1tbxAYtJKktrr9AeWPrYk5RitJUkEGrSRJBRm0kiQVZNBKklSQQStJUkEGrSRJBRm0kiQV5Hm0kqSudbpCWJzffnmvEGbQSpImweCcPLuOJUkqyKCVJKkgg1aSpIIMWkmSCvJgKElbOfTcm9m4abTr5bu9ldrcObO54+yjX2xZ0kAyaCVtZeOm0SK3Pev1vU2lQWDQqiudzp3rxFMAJKniGK26kplbPRYsvaHtdENWkn7BPVptwbE5Seotg1ZbeGHoA+xSol0A7irQsiT1N4NWW3hm7QoPgpGkHjJoJW1ll4XLOOTyZQXaBej9Dzmpnxm0krZiz4bUOwattlLiy3DunNk9b1OSBoFBqy1MZi9maNmNRfZ6JGkm8TxaSZIKMmglSSrIoJUkqSCDVpKkgjwYSl3pdFOBOL/98l7vWJIqU96jjYh9I2J1RKyNiHsi4k/q6a+IiC9FxHfr//7y1MtVUzrdPMCbCkjSxHrRdfw88IHMXAgcAbw/Ig4GlgH/nJkHAP9cv5Ykabsy5aDNzMcy87b6+TPAWmA+cAJweb3Y5cDbprouSZIGTU8PhoqIIeDXgW8Cv5KZj0EVxsAevVyXJEmDoGdBGxE7A58F/jQzfzKJ950WESMRMbJhw4ZelSNJUl/oSdBGxGyqkL0yMz9XT/5hROxVz98L+FG792bmxZk5nJnD8+bN60U5kiT1jV4cdRzAJcDazPxwy6zrgVPq56cAn5/quiRJGjS9OI/29cC7gLsi4vZ62oeAFcA1EbEEeBD4/R6sS5KkgTLloM3MNUD7qxnAm6bavqRmeLtEqTe8MpRmnE5XserEC2xszdslSr3jtY4147S7UtWCpTd4FStJjTBoJUkqyKCVJKkgx2g10A4992Y2bhrtatluD+6ZO2c2d5x99FTKkqSfM2g10DZuGu35gTgljraVtP2y61iSpILco9VA22XhMg65vLd3YNxlIYCnq0jqDYNWA+2ZtSvsOpbU1wxaSWqYF1mZ2QxaDbxe74F6mUBNt3bB6RW3Zg6DVgOt2y8iv7QkNcWg1YzTqRsuzm+/vN1wkkoyaDXjGJyS+onn0UqSVJBBK0lSQQatJEkFGbSSJBXkwVCSujLRRRXaHdHtQWlSxaCV1BWDU3pxDFpJmiaTuX8yeA/lmcKglaRpUuL+yeCNMPqdB0NJklSQe7SSZqTJ3BFnusafS9w/uWoXvIdy/zJoJc1I/XhHnBL3Twa7jvudXceSJBVk0EqSVJBBK0lSQQatJEkFeTCUpIHmRSDUyWSOPIdyR58btJIGmheBUCedgnO6jz6361iSpIIMWkmSCjJoJUkqyDFaSQNt0C5rWGLsd+6c2T1vU71j0EoaaIN0WcPJ1Nn05SLVO3YdS5JUkHu0kqSBV+J86l6dS23QSpIGXonzqXs1fGDXsSRJBblHK0kaeCWOPu/VkecGraSB5ykzKnH0ea8+VzMqaPvlAtKSpo+nzKjfzagx2sxs+1iw9Ia20yVJKm1G7dFKkrZfvR5C6NXwQfGgjYhjgI8Bs4BPZeaK0uuUJG1f+nkIoWjXcUTMAv4GeCtwMLA4Ig4uuU5JkvpJ6T3aw4B1mfk9gIi4CjgBuHeqDZe4Cgj07kogkqRmTXSAbJy/9bRSx+6UDtr5wEMtrx8GDu+08JNPPsnKlSu7avh1m5/kiIN2m1Jx7dzyvSdZufLRnrcrqXnHvORJVq7c0HQZXRmkWvvVZZddNqnlu82fySodtO1+TmzxkyEiTgNOA5g/f37hciSp/3zlK19pP/2htpM56qijClajXouSp7lExOuAczLzLfXrswAy8y/bLT88PJwjIyNdtV1qMNvz7KSZy3/fKiUibs3M4XbzSu/R/itwQETsDzwCnAy8oxcND9rNniVJ26eiQZuZz0fE6cA/Up3ec2lm3tOLtgfpZs+SpO1X8fNoM/Mm4KYSbXt9U0lSvxvYK0P188nJkiSNGdigbadfzpmSJGnMjApag1OS1G9m1N17JEnqNzNqj1aSxnQaSnIYSdPNoJU0Ixme6hd2HUuSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0PaJVatWsWjRImbNmsWiRYtYtWpV0yVJknrAK0P1gVWrVrF8+XIuueQSjjzySNasWcOSJUsAWLx4ccPVSZKmIvrpMmXDw8M5MjLSdBnTbtGiRVx44YW84Q1v+Pm01atXc8YZZ3D33Xc3WJkkqRsRcWtmDredZ9A2b9asWTz33HPMnj3759NGR0fZaaed2Lx5c4OVSZK6MVHQOkbbBxYuXMiaNWu2mLZmzRoWLlzYUEWSpF4xaPvA8uXLWbJkCatXr2Z0dJTVq1ezZMkSli9f3nRpkqQp8mCoPjB2wNMZZ5zB2rVrWbhwIeedd54HQknSDOAYbYM63Zi6nX76/yRJ2pJjtH0qM7d6LFh6Q9vpkqTBZNBKklSQY7TT4NBzb2bjptGulx9admNXy82dM5s7zj76xZYlSZoGBu00eGHoA+xSol0A7irQsiSpVwzaafDM2hWsX3Fsz9vtds9XktQcx2glSSrIPdppUmLvc+6c2dteSJLUKIN2Gkym23ho2Y1FupklSc2w61iSpIIMWkmSCjJoJUkqyDHaBnW61nGcv/U0L8MoSYPJoG2Q4SlJM59dx5IkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJU0JSCNiIuiIj7IuLOiLguInZtmXdWRKyLiO9ExFumXqokSYNnqnu0XwIWZeargfuBswAi4mDgZODXgGOAj0fErCmuS5KkgTOloM3MmzPz+frlLcA+9fMTgKsy82eZ+X1gHXDYVNYlSdIg6uUY7X8AvlA/nw881DLv4XqaJEnblW3eJi8i/gnYs82s5Zn5+XqZ5cDzwJVjb2uzfNt7wkXEacBpAPvtt18XJUuSNDi2GbSZ+eaJ5kfEKcBxwJvyFzdYfRjYt2WxfYBHO7R/MXAxwPDwsDdolSTNKFM96vgYYClwfGb+tGXW9cDJEbFjROwPHAB8ayrrkiRpEG1zj3YbLgJ2BL4UEQC3ZOZ7M/OeiLgGuJeqS/n9mbl5iuuSJGngTCloM/NVE8w7DzhvKu1LkjTovDKUJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFTfX0HklTUJ8W17VfXBNG0qBwj1ZqUGa2fSxYekPb6ZIGj0ErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFbRD0wVI24tDz72ZjZtGu15+aNmN21xm7pzZ3HH20VMpS1JhBq00TTZuGmX9imN72mY3YSypWXYdS5JUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJUUE/u3hMRHwQuAOZl5hMREcDHgN8Ffgqcmpm39WJd0qDaZeEyDrl8WY/bBOjtHYEk9daUgzYi9gV+B3iwZfJbgQPqx+HA/6r/K223nlm7wtvkSduhXnQdfwQ4E8iWaScAV2TlFmDXiNirB+uSJGmgTCloI+J44JHMvGPcrPnAQy2vH66nSZK0Xdlm13FE/BOwZ5tZy4EPAUe3e1ubadlmGhFxGnAawH777betciRJGijbDNrMfHO76RFxCLA/cEd17BP7ALdFxGFUe7D7tiy+D/Boh/YvBi4GGB4ebhvGkiQNqhfddZyZd2XmHpk5lJlDVOH62sx8HLgeeHdUjgA2ZuZjvSlZkqTB0ZPTe9q4ierUnnVUp/e8p9B6JEnqaz0L2nqvdux5Au/vVdvSTNHr03Hmzpnd0/Yk9V6pPVpJ40zmHNqhZTf2/JxbSc3wEoySJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkFGbSSJBVk0EqSVJBBK0lSQQatJEkF7dB0AdL2LCI6zzt/62mZWbAaSSUYtFKDDE5p5rPrWJKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSrIoJUkqSCDVpKkggxaSZIKMmglSSoo+ulaqxGxAfhBgaZ3B54o0G4J1lrGINUKg1WvtZZhreWUqHdBZs5rN6OvgraUiBjJzOGm6+iGtZYxSLXCYNVrrWVYaznTXa9dx5IkFWTQSpJU0PYStBc3XcAkWGsZg1QrDFa91lqGtZYzrfVuF2O0kiQ1ZXvZo5UkqREDH7QRcWJEZEQc1DLtgIi4ISIeiIhbI2J1RPy7et6pEbEhIm5veRw8jfUuj4h7IuLOet2HR8SXI+I7LfVcWy/71xHx38e992+msdbNdT13RMRtEfFv6+lD9Tb/85Zld4+I0Yi4qH59TkR8cLpqnUTdm+p590bE30ZE4/8GOnwmTo+IdfV23r3pGsd0qPXK+vN7d0RcGhGzm64TOtZ6Sf25uDMiro2InZuuc0y7elvmXRgRzzZZX6sO23ZlRHy/5XvsNU3XCR1rjYg4LyLuj4i1EfGfixaRmQP9AK4BvgqcU7/eCbgfOL5lmUXAqfXzU4GLGqr1dcA3gB3r17sDewNfBobbLP9y4HvAvwH2B74P7DqN9T7b8vwtwFfq50PAA8C3W+a/D7h9bNsC5wAfbGg7T1T33fXzHYB/AX6viRq7+Ez8el3vemD3JmvsotbfBaJ+rALe18e1vrxlmQ8Dy5qudaJ66+fDwKdbP9f9WCuwEjip6fq6rPU9wBXAL9XT9yhZxw4MsPrX6OuBNwDXU325vxP4RmZeP7ZcZt4N3N1EjePsBTyRmT8DyMwnACKi7cKZ+ZOIWA5cVE/6H5n59HQU2sbLgR+3vN4ErI2I4cwcAd5O9aNn7yaKm8D4ugHIzOcj4uvAq6a/pC20/UwAj0Lnz0ZDJqwVICK+BezTQG3jdaoVgKg27BygXw5S6fTdMAu4AHgHcGJz5W1hUt9jDetU6/uAd2TmC/X0H5UsovFusyl6G/DFzLwfeCoiXgv8GnDbNt739nFdx3OKV1q5Gdi37q74eEQc1TLvypZ6LhibmJmrgF+m+iX+6Wmqc8ycup77gE8Bfz5u/lXAyRGxD7CZli/chm2rbiLipcCbgLumu7hxJvpM9JsJa627jN8FfLGR6rbUsdaIuAx4HDgIuLCpAsfpVO/pwPWZ+ViDtY030efgvLqL9iMRsWNTBbboVOsrqXJgJCK+EBEHlCxi0IN2MdWXPfV/F49fICKuq8eOPtcy+erMfE3LY9N0FJuZzwK/AZwGbACujohT69nvbKnnv7XUvw+wJ7B3A+NJm+p6DgKOAa6ILX+2fhH4HartfvU01zaRiep+ZUTcDnwNuDEzv9BYlWzzM9FXuqj148C/ZOZXGyhvCxPVmpnvoep5WUvVE9O4DvV+CPh9+ufHADDhtj2L6sfLbwKvAJY2VeOYCWrdEXguq6tDfRK4tHQhA/kAdqPqvvwB1TjWQ8CDwBLg8nHLDgNfrp+fSkNjtG3+hpOAf6DDGG29zGeBU4AVwAXTXN+z417/ENiDLcc6L6XaO9itddvSJ2O0neru18fYZ6Ll9Xr6ZIx2olqBs4G/px7z6rfH+O1aTzsKuKHp2iaod7T+t7W+frwArGu6ti637W/347Zt+c69DxiqpwWwseR6B3mP9iTgisxckJlDmbkv1cFC9wOvj4jjW5Z9aSMVjhMRB47rongNE9xEISLeShUQV1B1f54Y03iE9LhaDgJmAU+Om/VXwNLMHD+9L0xQd1+Y7GeiSZ1qjYg/ojrobHHWY15N61DrgxHxqnp+AP+e6gu3cR3q/URm7ll/vw0BP83Mpo8pmOhzsFc9P6iG9Ro/LmaCf19/D7yxnnYUVW4UM8gHQy2m2str9VmqgwaOAz4cER+l2pt5BviLluXeHhFHtrz+T5n59ZLF1nYGLoyIXYHngXVUXRrXUo3RjnVhP0H1N3yU6ii+BP5vRJxJdWDUG7dquYw5dTcrVL/6TsnMza29x5l5D3DPNNXTrW3W3Ufafibq0w3OpBo2uDMibsrMP2qwTuj8+X2c6svrG/U2/lxm/lljVVba1fpe4LqIeDnV5+IOqqPl+0GnbduPOtV6TUTMo9q2t1Nt76Z1qvV5qu/c/wI8CxT9t+WVoSRJKmiQu44lSep7Bq0kSQUZtJIkFWTQSpJUkEErSVJBBq0kSQUZtJIkFWTQSpJU0P8HygEFzo7FNDEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_validate, RepeatedKFold # import sklearn methods\n",
    "\n",
    "rng = np.random.RandomState(1) # make sure the results are repeatable\n",
    "feature_names = nX.columns.tolist() # write the column names to a list\n",
    "# cross_validate takes the particular model, in this case linear regression which we \n",
    "# instantatiated earlier,\n",
    "# and undertakes a number of runs according the method specified by cv=\n",
    "# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times, \n",
    "# giving 25 runs\n",
    "# return_estimator=True returns the fitting data for each run\n",
    "scores = cross_validate(linear, nX, Y, cv=RepeatedKFold(n_splits=5, n_repeats=5), \n",
    "                        return_estimator=True,return_train_score=True)\n",
    "# take the results for each simulation (estimator), extract the coefficients for each run \n",
    "# and add them to a dataframe with columns being the feature names\n",
    "coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)\n",
    "# plot the descriptive statics of the coefficients in a box and whisker plot to show \n",
    "# variability\n",
    "ax = coefs.plot(kind='box',figsize=(10,5))\n",
    "plt.title('Linear Regression')\n",
    "plt.axhline(y=0, color='.5')\n",
    "plt.subplots_adjust(left=.3)\n",
    "print('Testing score is',\n",
    "      round(np.mean(scores['test_score']),4),\"±\",round(np.std(scores['test_score']),4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regularization \n",
    "\n",
    "We now investigate regularization techniques for Linear Regression, to reduce the variance of the model. We will concentrate of ridge regularization, though other possible techniques are lasso and elastic-net regularizations.\n",
    "\n",
    "To use Ridge regularization (which adds a penalty term which is proportional to the sum of the squares of the coefficients), we need to find the optimal value of the tuning parameter alpha. Here we will first graphically compare the training and testing scores to determine the value of alpha for which we obtain the maximum value of the testing score. To generate the figure we create an array of alpha values, which in this case are logarithmically distributed, and perform a Ridge regularization for each, and store the testing and training scores ( $R^2$ ). Then we plot these against alpha. From the figure we see the optimal value occurs at alpha approximately 20."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEaCAYAAADzDTuZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXhU5Z3/8feXGCBqJApWloAbWl0UQoAYo6i1rk+Aj/zUKtpF21q1tmq7rWzhuqxF7G5RaqlYWsrWp1qVUhVKLRitLraiAuEpPEmhrNUEXCOVBzEowe/vj5nEIZkkM5kzmTPJ53VduTLnPve55zs34XzPuc+Z+5i7IyIiXVu3TAcgIiKZp2QgIiJKBiIiomQgIiIoGYiICEoGIiKCkoGkmZnNMrPvt7Lezey4jowpaGZWFP0ch7Rz+8+b2aY0xPUlM3s+6HalczJ9z0BSYWZvAscAB4APgOeAW9z9gwS3d+B4d9+StiDTzMyKgP8Fct29vqvGINlNZwYShIvd/XBgODACmJTheA5iEaH8W2/v2YRI0EL5H0Syk7u/A1QQSQoAmNkjZvbDmOUJZrbdzLaZ2Vdjtzez3mb2BzPbbWbLzeyHZvZKzPoTzOwFM/uHmW0ysytbisXMFpvZf5rZEuBD4LNm1svMHoy+f020/Zxo/Rwzu8/M3jOz/zWzW2KHfszsTTM7N6b9yWb2mxbe+ytmttHM9pjZVjO7KWbdWWZWbWbfM7N3gIcbyqLrrzKzD2J+PjKzxdF1F5rZqmj/vG1mk2Pe9s/R3zuj2400sy836b/Tov26K/r7tCb9dbeZLYnG/byZ9Wmpf6XzUTKQwJhZf2AMEHfIx8xGA7cD5wHHA+c2qTIT2Av0Ba6L/jRsexjwAvAE8BngauDnZjaklZDGAzcC+cDfgUeBeuA4Imcw5wNfi9a9IRr7cKAUGJvAR27Ju8BFwBHAV4DpZlYas74vcBTwz9H4Grn7b9398OiZVj9gK/BkdPVe4FqgALgQuNnMGuI8M/q7ILr9a7HtmtlRwB+BGUBv4CfAH82sd0y1a6LxfgboTuTfSroIJQMJwnwz2wO8TWRH+IMW6l0JPOzu69x9LzC5YUX0CP1y4Afu/qG7byCy825wEfCmuz/s7vXuvhJ4Griilbgecff10TH0o4js7L/t7nvd/V1gOjAuJrb73b3a3d8HpibVAzHc/Y/u/jePeBl4Hvh8TJVPop/zI3evi9dGdFjrCWCxu/8y2u5id1/r7p+4exWRJPGFBMO6ENjs7o9F++9J4A3g4pg6D7v7X6MxzSXmDE86PyUDCcJYd88HzgJOAFoaXuhHJGE0+HvM66OBQ5qsj339z8ApZraz4Qf4EpGj7JY03T4X2B6z/S+JHAXHiy32dVLMbIyZvR4dztoJXMDBfVLr7vvaaOY/iZzR3BbT7ilm9j9mVmtmu4Cv03JfN9WPg/ub6HJhzPI7Ma8/BA5PsG3pBJQMJDDRo+BHgB+3UGU7MCBm+diY17VEhnD6x5TF1n0beNndC2J+Dnf3m1sLqcn2HwF9YrY/wt0bhpm2t/LeEBmiOTRmOW4SMrMeRM5Yfgwc4+4FwELAWogrXhvjiAyDXeHu+2NWPQEsAAa4ey9gVky7bd0WuI1IQox1LFDTxnbSRSgZSNB+CpxnZvGGGOYCXzazwWZ2KDHDSe5+AHgGmGxmh5rZCUTGxxs8C/yLmY03s9zoz8lmdmIiQbn7diLDNfeZ2RFm1s3MPmdmDcMsc4FvmVmhmRUA32vSxGpgXPR9y2h5eKo70INocjOzMUSuTSTEzEYADxA526ptsjof+Ie77zOzciJj/A1qiQw/fbaFphcS6b9rzOwQM7sKGEykX0WUDCRY0R3Yr4FmXzRz90VEksVLRC4yv9Skyi1ALyLDFY8RGRP/KLrtHiI71XFEjnLfAe4hsuNN1LVEdtYbgPeBp4B/iq77byLJogpYRWTnWU/k+xNEP8/notvdReQovZlonLcRSS7vE9lhL0gixkuBI4FXYu4oWhRd9w1gSvT6zJ3R92h43w+JDC0tiQ6Dndokrh1Errt8F9gB/Adwkbu/l0Rs0onpS2cSWmZ2D9DX3a9rs3Lw7z0GmOXuTYdWRDolnRlIaES/R1BiEeXA9cC8DnrvPDO7IDqEUkhkCKtD3lskDJQMJEzyiVw32EtkCOQ+4Pcd9N5GZPjnfSLDRBuJDMWIdAkaJhIREZ0ZiIiIkoGIiBD5xmeo9OnTx4uKijIdhohIVlmxYsV77n50e7cPXTIoKiqisrIy02GIiGQVM2s63UhSNEwkIiJKBiIiomQgIiIoGYiICEoGIiKCkoGIiKBkICIiKBmIiAhKBiIiQoLJwMxGm9kmM9tiZhNbqXeFmXn0sYBEHxH4qJmtNbONZjYpqMBFRCQ4bSYDM8sBZgJjiDwz9WozGxynXj6Rx/0tjSn+ItDD3YcCJwE3mVlR6mGLiEiQEjkzKAe2uPtWd/8YmEPkOa1N3Q3cC+yLKXPgMDM7BMgDPgZ2pxayiIgELZFkUAi8HbNcHS1rZGYjgAHu/myTbZ8i8tSq7cBbwI/d/R9N38DMbjSzSjOrrK2tTSZ+EREJQCLJwOKUNT4ezcy6AdOB78apVw4cAPoBA4HvmtlnmzXmPtvdy9y97Oij2z0Dq4iItFMiU1hXAwNilvsD22KW84FiYLGZAfQFFpjZJcA1wHPuvh9418yWAGXA1gBiFxGRgCRyZrAcON7MBppZd2AcsKBhpbvvcvc+7l7k7kXA68Al7l5JZGjobIs4DDgVeCPwTyEiIilpMxm4ez1wC1ABbATmuvt6M5sSPfpvzUzgcGAdkaTysLtXpRiziIgEzNy97VodqKyszPWkMxGR5JjZCncva+/2+gayiIgoGYiIiJKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiKCkoGIiJDYFNYdam3NLk6f+hITRg1i7IjCtjeIMX9VDdMqNrFtZx39CvKyto0wxBCWNsIQQ1g+h0g65UyePDnTMRzkh/c9MNlPOJeX/1pL/yPzOOGfjkhou/mrapj0zFr+8eHHAOzZV5+VbYQhhrC0EYYYwvI5Gtq5/tFKfvjsBn5XWU3vw7ontb10bnfdddf2yZMnz27v9qEdJqrbf4BpFZsSrj+tYhN1+w9kfRthiCEsbYQhhiDaCCKGhoRSs7MOB2p21jHpmbXMX1WTcBsN7Zw+9SUGTvwjp099KentpfMK3TBRrG0761Kum21thCGGsLQRhhiCaCOIGFpLKIkONzUklIZ2GhIKoCEvCe+ZAUC/gryU62ZbG2GIISxthCGGINoIIoZ0J5REBXWGIuET2mSQl5vDhFGDEq4/YdQg8nJzsr6NMMQQljbCEEMQbQQRQ2dKKBJOoRwmKmzHqWdD3VROX8PQRhhiCEsbYYghLJ9jwqhBBw3xQPsSSk2cHX9HJxQJJz32UiRLBHF7a7yE8qPLhibczulTX4qbUAoL8lgy8eyE49A1h+Cl+tjLUJ4ZiEhzY0cUprTTDMMZSlAXsSV4SgYiXUimE0oQd0VJeigZiEhSUkkouuYQXqG9m0hEOp8g7oqS9FAyEJEOE8RttpIeGiYSkQ4TxEVsSQ8lAxHpUKlexAbdnpoOSgYiklV0e2p6JHTNwMxGm9kmM9tiZhNbqXeFmbmZlcWUlZjZa2a23szWmlnPIAIXka5JU2KkR5tnBmaWA8wEzgOqgeVmtsDdNzSplw/cBiyNKTsE+A0w3t3XmFlvYH+A8YtIF6PbU9MjkTODcmCLu29194+BOcClcerdDdwL7IspOx+ocvc1AO6+w90PxNlWRCQhuj01PRJJBoXA2zHL1dGyRmY2Ahjg7s822fZfADezCjNbaWb/Ee8NzOxGM6s0s8ra2tokwheRrka3p6ZHIheQLU5Z4+x2ZtYNmA58uYX2zwBOBj4EXoxOpvTiQY25zwZmQ2SiuoQiF5EuSbenpkciyaAaGBCz3B/YFrOcDxQDi80MoC+wwMwuiW77sru/B2BmC4FS4KBkICKSjCBuT5WDJTJMtBw43swGmll3YBywoGGlu+9y9z7uXuTuRcDrwCXuXglUACVmdmj0YvIXgA3N30JERDKpzWTg7vXALUR27BuBue6+3symRI/+W9v2feAnRBLKamClu/8x9bBFRCRIeriNiEgnkOrDbTRRnYiIKBmIiIiSgYiIoInqRKSL0synB1MyEJEuRzOfNqdhIhHpcjTzaXNKBiLS5Wjm0+aUDESky9HMp80pGYhIl6OZT5vTBWQR6XI082lzSgYi0iVp5tODaZhIRESUDERERMlARERQMhAREZQMREQEJQMREUHJQEREUDIQERGUDEREBCUDEeloVXNhejFMLoj8rpqbmTbkIJqOQkQ6TtVc+MNtsD86VfSutyPLACVXdlwb0ozODEQkOakclb845dOdeIP9dZHyjmxDmtGZgYgkLtWj8l3VyZWnqw1pRmcGIpK4VI/Ke/VPrjxdbUgzSgYiXUmqF15TPSo/507IbfI0sdy8SHmigmhDmlEyEOkqGoZ4dr0N+KdDPMkkhFSPykuuhItnQK8BgEV+XzwjuQu/QbQBuiOpCXP3tiuZjQbuB3KAX7n71BbqXQH8DjjZ3Stjyo8FNgCT3f3Hrb1XWVmZV1ZWtlZFRNpjenE0ETTRawD8+7rE2mh6zQAiR+Xt2RlnUkCfY/6qmtA8Lc3MVrh7WXu3b/PMwMxygJnAGGAwcLWZDY5TLx+4DVgap5npwKL2BikiZH6IB4I7Ks+0AO5Imr+qhknPrKVmZx0O1OysY9Iza5m/qibYWDtIIncTlQNb3H0rgJnNAS4lcqQf627gXuD22EIzGwtsBfamHK1IVxXEvfW9+rdwZpDkhdeSK7Nv599UAIlxWsUm6vYfOKisbv8BplVsysrHaSZyzaAQiP0Lqo6WNTKzEcAAd3+2SflhwPeAu1p7AzO70cwqzayytrY2ocBFskqqR/VB3FuvC6+fCuCOpG0765IqD7tEkoHFKWu80GBm3YgMA303Tr27gOnu/kFrb+Dus929zN3Ljj766ARCEskiQVy41RBPsAJIjP0K8pIqD7tEhomqgQExy/2BbTHL+UAxsNjMAPoCC8zsEuAU4AozuxcoAD4xs33u/rMgghfpMFVzI0fhu6ojR4/n3Jn4TrS1o3oN8WRGQx+0998UmDBqEJOeWXvQUFFebg4TRg0KOtoOkUgyWA4cb2YDgRpgHHBNw0p33wX0aVg2s8XA7dG7iT4fUz4Z+ECJQDpcKjvyhu0z/a3bc+6Mf/dLVxziCUqKibHhukBY7iZKVZvJwN3rzewWoILIraUPuft6M5sCVLr7gnQHKV1YpnfkkPqRfRBH9QEcyUrwxo4ozNqdf1MJfc+gI+l7BgFJdScahjaCuBc8iHvrJxcQc5kshsHknW1v31nuzZdQS/v3DDrcttWZneM8DG0EsX2qFyzD0EYQd9AEMUQThm/diqRZ+JJBNu+8gmgjiBjCMk1wqm2EYUcOwdySWXJl5Exk8s7IbyUCCZkQJoOobNx5BdFGWI6Gw9BGmHbkOrKXTi7czzPItp1XEG0EdTSc6gXLMLQRxB00QV141S2Z0smF98wAOn6O8zC0EZaj4TC0EdQRuYZoRNoU3mSQjTuvINoIy7BGmNrQjlwk7cJ3a2m/Q7xywgnZeStkUG0EEYOIdCmp3loavmSg7xmISLYI0YFbqskg3BeQRUTCKohvt4dIeK8ZiIiEWRC3gYeIkoGISHsEcRt4iCgZiIi0RxC3gYeIkoGISHt0sifHKRmIiLRHJ5umRHcTiYi0VyeapkRnBiIiomQgIiJKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiKCkoGIiJBgMjCz0Wa2ycy2mNnEVupdYWZuZmXR5fPMbIWZrY3+PjuowEVEJDhtzlpqZjnATOA8oBpYbmYL3H1Dk3r5wG3A0pji94CL3X2bmRUDFUBhUMGLiEgwEjkzKAe2uPtWd/8YmANcGqfe3cC9wL6GAndf5e7boovrgZ5m1iPFmEVEJGCJJINC4O2Y5WqaHN2b2QhggLs/20o7lwOr3P2jpivM7EYzqzSzytra2gRCEhGRICXycBuLU+aNK826AdOBL7fYgNkQ4B7g/Hjr3X02MBugrKzM49UREels5q+qYVrFJrbtrKNfQR4TRg1i7IjMjKQnkgyqgQExy/2BbTHL+UAxsNjMAPoCC8zsEnevNLP+wDzgWnf/WzBhi4hkt/mrapj0zFrq9h8AoGZnHZOeWQuQkYSQyDDRcuB4MxtoZt2BccCChpXuvsvd+7h7kbsXAa8DDYmgAPgjMMndl6QhfhGRrDStYlNjImhQt/8A0yo2ZSSeNpOBu9cDtxC5E2gjMNfd15vZFDO7pI3NbwGOA75vZqujP59JOWoRkSy3bWddUuXplsgwEe6+EFjYpOzOFuqeFfP6h8APU4hPRKRT6leQR02cHX+/grwMRKNvIIuIZMSEUYPIy805qCwvN4cJowZlJJ6EzgxERCRYDReJs+luIhERSYOxIwoztvNvSsNEIiKiZCAiIkoGIiKZUzUXphfD5ILI76q5GQtF1wxERDKhai784TbYH729dNfbkWWAkis7PBydGYiIZMKLUz5NBA3210XKM0DJQEQkE3ZVJ1eeZkoGIiKZ0Kt/cuVppmQgIpIJ59wJuU2mnsjNi5RngJKBiEgmlFwJF8+AXgMAi/y+eEZGLh6D7iYSEcmckisztvNvSmcGIiKiZCAiIkoGIiKCkoGIiKBkICIiKBmIiAhKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICAkmAzMbbWabzGyLmU1spd4VZuZmVhZTNim63SYzGxVE0CIiEqw2n2dgZjnATOA8oBpYbmYL3H1Dk3r5wG3A0piywcA4YAjQD/iTmf2Lux8I7iOIiEiqEjkzKAe2uPtWd/8YmANcGqfe3cC9wL6YskuBOe7+kbv/L7Al2p6IiIRIIsmgEHg7Zrk6WtbIzEYAA9z92WS3jW5/o5lVmlllbW1tQoGLiEhwEkkGFqfMG1eadQOmA99NdtvGAvfZ7l7m7mVHH310AiGJiEiQEnkGcjUwIGa5P7AtZjkfKAYWmxlAX2CBmV2SwLYiIhICiZwZLAeON7OBZtadyAXhBQ0r3X2Xu/dx9yJ3LwJeBy5x98povXFm1sPMBgLHA8sC/xQiIpKSNs8M3L3ezG4BKoAc4CF3X29mU4BKd1/QyrbrzWwusAGoB76pO4lERMLH3JsN4WdUWVmZV1ZWZjoMEZGsYmYr3L2s7Zrx6RvIIiKiZCAiIkoGIiKCkoGIiKBkICIiKBmIiAhKBiIigpKBiIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICEoGIiJCYo+9zLj9+/dTXV3Nvn37Mh2KtKFnz57079+f3NzcTIci0iXMX1XDtIpNdO973EmptJMVyaC6upr8/HyKioqIPmdZQsjd2bFjB9XV1QwcODDT4Yh0evNX1TDpmbXU7U/9AZJZMUy0b98+evfurUQQcmZG7969dQYn0kGmVWwKJBFAliQDQIkgS+jfSaTjbNtZF1hbWZMMRETkYP0K8gJrS8kgATt27GD48OEMHz6cvn37UlhY2Lj88ccfJ9TGV77yFTZt2tRqnZkzZ/L4448HEbKIdAETRg0iLzcnkLay4gJyshqurm/bWUe/gjwmjBrE2BGF7W6vd+/erF69GoDJkydz+OGHc/vttx9Ux91xd7p1i59fH3744Tbf55vf/Ga7Y0yntj6biGRGw35tWsUmtqfYVqf7391wdb1mZx0O1OysY9Iza5m/qibw99qyZQvFxcV8/etfp7S0lO3bt3PjjTdSVlbGkCFDmDJlSmPdM844g9WrV1NfX09BQQETJ05k2LBhjBw5knfffReAO+64g5/+9KeN9SdOnEh5eTmDBg3i1VdfBWDv3r1cfvnlDBs2jKuvvpqysrLGRBVrwoQJDB48mJKSEr73ve8B8M4773DppZdSUlLCsGHDWLp0KQD33nsvxcXFFBcX88ADD7T42RYtWsTIkSMpLS3lqquuYu/evYH3qYgkZ2zOEpb0uI2htjWldjpdMoh3db1u/wGmVbQ+RNNeGzZs4Prrr2fVqlUUFhYydepUKisrWbNmDS+88AIbNmxots2uXbv4whe+wJo1axg5ciQPPfRQ3LbdnWXLljFt2rTGxPLAAw/Qt29f1qxZw8SJE1m1alWz7f7v//6PhQsXsn79eqqqqpg0aRIQOfM477zzqKqqYsWKFZx44oksW7aMxx9/nGXLlvHaa6/x85//nKqqqmafLTc3l6lTp/Liiy+ycuVKSkpKuP/++4PqRhFpj6q58IfbYNfbKTfV6ZJBS1fXg7zqHutzn/scJ598cuPyk08+SWlpKaWlpWzcuDFuMsjLy2PMmDEAnHTSSbz55ptx277sssua1XnllVcYN24cAMOGDWPIkCHNtjvqqKPo1q0bN9xwA/PmzeOwww4DYPHixdx0000AHHLIIRxxxBH85S9/4fLLL+fQQw8lPz+fsWPH8sorrzT7bK+++iobNmzgtNNOY/jw4Tz++OMtxi0iHeTFKbA/mH1bp7tm0K8gj5o4O/4gr7rHatjRAmzevJn777+fZcuWUVBQwL/927/Fvee+e/fuja9zcnKor6+P23aPHj2a1XH3NmPKzc2lsrKSF154gTlz5vCLX/yC559/Hmh+62dr7cV+Nndn9OjRPPbYY22+v4h0kF3VgTWV0JmBmY02s01mtsXMJsZZ/3UzW2tmq83sFTMbHC3PNbNHo+s2mtmkwCJvQbyr63m5OUwYNSjdb83u3bvJz8/niCOOYPv27VRUVAT+HmeccQZz584FYO3atXHPPPbs2cPu3bu56KKLmD59euNQ0r/+678ya9YsAA4cOMDu3bs588wzmTdvHnV1dXzwwQf8/ve/5/Of/3yzNk877TRefvlltm6NjEvu3buXzZs3B/75RCQJvfoH1lSbycDMcoCZwBhgMHB1w84+xhPuPtTdhwP3Aj+Jln8R6OHuQ4GTgJvMrCig2OMaO6KQH102lMKCPAwoLMjjR5cNTeluokSVlpYyePBgiouLueGGGzj99NMDf49bb72VmpoaSkpKuO+++yguLqZXr14H1dm1axcXXnghw4YN4+yzz+YnP4n8c/zsZz+joqKCoUOHUlZWxhtvvEF5eTlXX301J598Mqeeeio333wzQ4cObfa+xxxzDA8++CBXXXUVw4YN47TTTuOvf/1r4J9PRJJwzp2QG8yoh7U17GBmI4HJ7j4qujwJwN1/1EL9q4Fr3X1M9PU1wP8DegGvAae6+z9aer+ysjKvrKw8qGzjxo2ceOKJCX+ozqy+vp76+np69uzJ5s2bOf/889m8eTOHHBKeET/9e4l0oKq58OIUyqZtpHLbgXZPAZDIHqQQiL1UXQ2c0rSSmX0T+A7QHTg7WvwUcCmwHTgU+PfWEoG07YMPPuCcc86hvr4ed+eXv/xlqBKBiHSwkiuh5EpWfMdWpNJMInuReJmm2emEu88EZprZNcAdwHVAOXAA6AccCfzFzP7k7gfdEGtmNwI3Ahx77LFJfYCupqCggBUrUvo3FxFpJpELyNXAgJjl/sC2VurPAcZGX18DPOfu+939XWAJUNZ0A3ef7e5l7l529NFHJxa5iIgEJpFksBw43swGmll3YBywILaCmR0fs3gh0HCbyVvA2RZxGHAq8EbqYYuISJDaHCZy93ozuwWoAHKAh9x9vZlNASrdfQFwi5mdC+wH3icyRASRu5AeBtYRGW562N2r0vA5REQkBQldeXT3hcDCJmV3xrz+VgvbfUDk9lIREQmxTjcdRToEMYU1wEMPPcQ777zTuJzItNYiIh2hc96TGL3vll3VkW/onXNn5PardkpkCutEPPTQQ5SWltK3b18gsWmtM6G+vl63q4p0MZ3vzOCgWfw88vsPt0XK0+DRRx+lvLyc4cOH841vfINPPvmE+vp6xo8fz9ChQykuLmbGjBn89re/ZfXq1Vx11VWNZxSJTGu9efNmTjnlFMrLy/n+979PQUFBsxj27NnDmDFjGDZsGMXFxTz11FMALF26lJEjRzJs2DBOOeUUPvzwQ+rq6rjuuusYOnQopaWl/PnPfwbgV7/6FePGjeOiiy5qnERv6tSplJeXU1JSctB03CLS+XS+ZBBvFr/9dZHygK1bt4558+bx6quvNu7U58yZw4oVK3jvvfdYu3Yt69at49prr21MAg1JIXayOmh5Wutbb72V22+/nWXLlnHMMcfEjWPhwoUUFRWxZs0a1q1bx3nnnce+ffsYN24cM2fOZM2aNTz//PP06NGDGTNm0L17d9auXctjjz3G+PHjG4e6XnvtNR577DFeeOEFFi5cyFtvvcXSpUtZvXo1r776auMzFUSk8+l8yaClWfwCnN2vwZ/+9CeWL19OWVkZw4cP5+WXX+Zvf/sbxx13HJs2beJb3/oWFRUVzeYOiqelaa2XLl3K5ZdfDsA111wTd9uSkhKee+45Jk6cyJIlS+jVqxcbN27k2GOPpbS0FIBevXqRk5PDK6+8wvjx4wEYMmQI/fr1Y8uWLQCcf/75HHnkkQA8//zzLFq0iBEjRlBaWsqWLVs0F5FIJ9b5BoZ79Y//oIcAZ/dr4O589atf5e677262rqqqikWLFjFjxgyefvppZs+e3WpbiU5rHc+JJ55IZWUlCxcuZMKECVx00UWMHj262XTVDTG3pOmU1XfccQfXX399wnGISPbqfGcG8Wbxy82LlAfs3HPPZe7cubz33ntA5K6jt956i9raWtydL37xi9x1112sXLkSgPz8fPbs2ZPUe5SXlzNv3jwA5syZE7dOTU0Nhx9+OOPHj+c73/kOK1euZMiQIfz9739vfO/du3dz4MABzjzzTB5//HEgMqHc9u3bOe6445q1OWrUKB588MHGR1tWV1c3fk4R6Xw635lBw11DAd5N1JKhQ4fygx/8gHPPPZdPPvmE3NxcZs2aRU5ODtdffz3ujs5quGMAAAXLSURBVJlxzz33AJFbSb/2ta+Rl5fHsmXLEnqPGTNmMH78eO655x4uuOCCuENODY/A7NatG927d2fWrFn06NGDJ598kptvvpl9+/aRl5fHSy+9xK233spNN93E0KFDyc3N5de//nWz6xcAF1xwAW+88QannnoqEElkTzzxBH369Emhx0QkrNqcwrqjaQrrg+3du5dDDz0UM+M3v/kN8+bN4+mnn850WK3qyv9eIpliZivcvdncb4nqfGcGnczy5cv59re/zSeffMKRRx4Z2u8miEh2UzIIubPOOqvxC28iIumSNReQwzacJfHp30kkO2VFMujZsyc7duzQjibk3J0dO3bQs2fPTIciIknKimGi/v37U11dTW1tbaZDkTb07NmT/v2D/06HiKRXViSD3NxcBg4cmOkwREQ6rawYJhIRkfRSMhARESUDEREJ4TeQzWwP0BGP/+oF7Erzdm3VbWl9MuVNy5ou9wHSPalQe/symW0Tqaf+DK4/W1ufSL8lUtYRfdlSHEFvl4n+bLo8yN3z2w61Be4eqh+gsoPeZ3a6t2urbkvrkylvWhZnOe392d6+TGbbROqpP4Prz9bWJ9JviZTp/3pq/Rn032ZXHib6Qwds11bdltYnU960rL2fKxWpvGei2yZST/0ZXH+2tj6RfkumLN3C/H+9pXUd/rcZxmGiSk9hsiU5mPozWOrP4Kgvg5Vqf4bxzKD1p8BIstSfwVJ/Bkd9GayU+jN0ZwYiItLxwnhmICIiHUzJQERElAxERCTLkoGZjTWz/zaz35vZ+ZmOJ9uZ2WfN7EEzeyrTsWQjMzvMzB6N/k1+KdPxZDv9PQYr2f1lhyUDM3vIzN41s3VNykeb2SYz22JmE1trw93nu/sNwJeBq9IYbugF1J9b3f369EaaXZLs18uAp6J/k5d0eLBZIJn+1N9j25Lsz6T2lx15ZvAIMDq2wMxygJnAGGAwcLWZDTazoWb2bJOfz8Rsekd0u67sEYLrT/nUIyTYr0B/4O1otQMdGGM2eYTE+1Pa9gjJ92dC+8sOe56Bu//ZzIqaFJcDW9x9K4CZzQEudfcfARc1bcPMDJgKLHL3lemNONyC6E9pLpl+BaqJJITVZNmQa0dJsj83dGx02SeZ/jSzjSSxv8z0H3Ahnx5ZQeQ/V2Er9W8FzgWuMLOvpzOwLJVUf5pZbzObBYwws0npDi6LtdSvzwCXm9kvyMw0C9kqbn/q77HdWvr7TGp/meknnVmcsha/BefuM4AZ6Qsn6yXbnzsAJdW2xe1Xd98LfKWjg+kEWupP/T22T0v9mdT+MtNnBtXAgJjl/sC2DMXSGag/00P9Giz1Z7AC6c9MJ4PlwPFmNtDMugPjgAUZjimbqT/TQ/0aLPVnsALpz468tfRJ4DVgkJlVm9n17l4P3AJUABuBue6+vqNiymbqz/RQvwZL/RmsdPanJqoTEZGMDxOJiEgIKBmIiIiSgYiIKBmIiAhKBiIigpKBiIigZCByEDN708z6pFpHJNsoGYiIiJKBdF1mNt/MVpjZejO7scm6IjN7wyJPMqsys6fM7NCYKrea2UozW2tmJ0S3KTezV81sVfT3oA79QCIpUDKQruyr7n4SUAbcZma9m6wfBMx29xJgN/CNmHXvuXsp8Avg9mjZG8CZ7j4CuBP4r7RGLxIgJQPpym4zszXA60RmfTy+yfq33X1J9PVvgDNi1j0T/b0CKIq+7gX8LvpIwunAkHQELZIOSgbSJZnZWUQe/DHS3YcBq4CeTao1nbgrdvmj6O8DfPpckLuB/3H3YuDiOO2JhJaSgXRVvYD33f3D6Jj/qXHqHGtmI6OvrwZeSaDNmujrLwcSpUgHUTKQruo54BAzqyJyRP96nDobgeuidY4icn2gNfcCPzKzJUBOkMGKpJumsBaJI/rQ8WejQz4inZ7ODERERGcGIiKiMwMREUHJQEREUDIQERGUDEREBCUDERFByUBERID/DxutjlcTT+qCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "rng = np.random.RandomState(1) # make sure the results are repeatable\n",
    "# create an array of 21 alpha values logarithmically distributed between 10**(-2) and 10**2\n",
    "alfas = np.logspace(-2,2,num=21) \n",
    "# create two arrays for storage of the same size as alfas, but filled with zeros\n",
    "ridge_training_score = np.zeros_like(alfas)\n",
    "ridge_rsquared = np.zeros_like(alfas)\n",
    "# loop over the values in the alfas array, at each loop the current value is alfa \n",
    "# and idx is incremented by 1, starting at 0\n",
    "for idx, alfa in enumerate(alfas): \n",
    "    ridge = Ridge(alpha=alfa) # instantatiate Ridge regularization with the current alfa\n",
    "    ridge.fit(X_train,Y_train) # train the model to our data set\n",
    "# calculate the training score and store in the array ridge_training_score\n",
    "    ridge_training_score[idx] = ridge.score(X_train,Y_train) \n",
    "    preds_linear = ridge.predict(X_test) # calculate the model prediction for the test data\n",
    "# calculate the correlation between the predicted and actual test data and store in the  \n",
    "# array ridge_rsquared\n",
    "    ridge_rsquared[idx] = r2_score(Y_test,preds_linear)  \n",
    "# plot the training score against alpha\n",
    "plt.scatter(alfas,ridge_training_score,label='Training score') \n",
    "# plot the testing score against alpha\n",
    "plt.scatter(alfas,ridge_rsquared,label='Testing score') \n",
    "plt.xscale('log') # make the x-axis a logarithmic scale\n",
    "plt.gca().set_xlim(left=.01, right=100); # fix the x-axis limits\n",
    "plt.title('Ridge regularization')\n",
    "plt.xlabel('alpha');\n",
    "plt.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now can investigate Ridge regularization using the optimal value of alpha $\\approx$ 20. Now we see significantly reduced variance in the coefficients and that the most important variables are `BMI`, `BP` and `S5`. Note that `S1`-`S4` have varied significantly, due to the collinearity (i.e., the variables are highly correlated)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing score is 0.489 ± 0.0538\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAeZklEQVR4nO3dfbRcd13v8feHEmuwj9BCnxvlQULLatHw4CVLCEItoAIuEAILWwwU0CLei9JA1BZprq3IwxVQKKa2uGiAxYPUFkt5KMUooim00HKUxwCFlIaCoYXITXu/94/ZwcnJnJOTnvmdmTl5v9aadWb23rP39+wzZz7z++3f7J2qQpIktXGPURcgSdJiZtBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQauJkeQtSf5olvmV5AELWdO42tu+WqAaHpvk5mEvK00ag1ZjI8mWJDuS3JHkliSXJDlo1/yqelFVvXqUNU4K99W+S/LkJJuS/Gf3+ntbkoP75h+Y5OIk3+/m/69R1qvJYdBq3PxqVR0EnAo8DHjFiOvZTXqG9n8z7PVpXg4FzgeOAZYDxwGv6Zt/HvBA4ERgFfDyJKcvcI2aQP6DayxV1S3Ah+gFLgBdC/f8vsd/kGRrkm8l+a3+5ye5T5K/71of/5bk/CSb+uY/OMmHk3w3yX8k+Y2Zakny8STrk/wT8EPgZ5IcmmRDt/1vdus/oFv+gCSvTfKdJF9NcnbXrX3Pu7m+ByS5Nsn2bp3v6qYnyeuT3NrN+2ySk2fYVy9I8qXu9708yTF98yrJi5J8Mcn3krw5Sebyd0ryvCRTSW5P8pUkL5xl2S1JXpHk8912/ibJT05b5mXd77M1yfP6pj85yWe6v+c3kpw3l/r2RVVdVlVXVdUPq+p7wNuAR/ct8pvAq6vqe1U11c0/c9h1aPExaDWWkhwHPBH40gzzTwd+H3gCvVbG46ct8mbgB8BRwBndbddzfwr4MHAZcF9gNfCXSU6apaTnAmcBBwNfAy4F7gQeQK/lfRrw/G7ZF3S1nwr8HPDUea7v1cDVwOH0Wllv7KafBvwi8CDgMOCZwG3TN5TkccCfAr8BHN1t753TFvsV4OHAKd1yvzzLvuh3a/fcQ4DnAa9P8nOzLP+cbt337+r+w755R9FrVR4LrAHenOTwbt4P6AXdYcCTgRcnGbRfSXJC1/070+3Zc/zdfhG4qVvn4fRaujf0zb8BmO01I/VUlTdvY3EDtgB3ALcDBXwUOKxv/iXA+d39i4EL+uY9qHvOA4ADgJ3Az/bNPx/Y1N1/JvCP07b9VuDcGer6OPAnfY/vB/wIWNo3bTVwTXf/Y8AL++Y9vqvtnndzfW8HLgKOm1bX44AvAI8C7jFtXv++2gD8Wd+8g7r9s6x7XMDKvvnvBtbezb/h3wEv7e4/Frh52t/3RX2PnwR8uW/ZHbv2UTftVuBRM2znDcDrG74WnwB8D3hQ9/j4bj/95LRltoz6/8bb+N9s0WrcPLWqDqb3xvtg4IgZljsG+Ebf46/13T8SuOe0+f33TwQe2d/KodfSOmqWuqY/fwmwte/5b6XXOh5UW//9u7O+lwMB/jXJTbu6yavqY8Cb6LXev53koiSHDNjWMfTtn6q6g17L99i+ZW7pu/9DemG8V0memORfui7p/6QXnjP9zWDPv9kxfY9vq6o7B9WR5JFJrkmyLcl24EV72c7dluRR9Ho7nl5VX+gm39H97N+/h9D7UCjNyqDVWKqqa+m1yv58hkW20mtl7HJC3/1t9Lphj+ub1r/sN4Brq+qwvttBVfXi2Uqa9vwfAUf0Pf+QqtrVjbh1lm3v8/qq6paqekFVHQO8kF439wO6eX9RVT9PrwvzQcAfDNjWt+iFOfDjrvP7AN+c5ffdqyQHAu+l9ze6X1UdBnyQ3oeCmUz/m31rjpu7DLgcOL6qDgXeMtN2uq7jO2a5PWeW3+lh3XZ+q6o+umt69Y7ZbqXXtb7LKXRdy9JsDFqNszcAT0hy6oB57wbOTPKQJPcCzt01o6ruAt4HnJfkXkkeTO/43i5XAA9K8twkS7rbw5Msn0tRVbWV3jHT1yY5JMk9ktw/yWP6antpkmOTHAacM5/1JXlGd8waet2ZBdzV1fzIJEvoHcP8L+CuAZu4DHheklO7cPzfwKeqasveftcky7rBUssGzP4J4EC6DzZJnkjvuPFsfifJcUnuDbwSeNfeaugcDHy3qv4rySOAGY+zVtXXuw9OM93eMeh53UCyq4CXVNXfD1jk7cAfJjm8e029gN6HQWlWBq3GVlVto/fmtseJF6rqH+gF8cfoDZj62LRFzqY3sOYW4G+BjfRajVTV7fQC4Vn0WlS3ABfSC425+k16QfN5euH3HnoDjaA3GvVq4LPAZ+i18u5kcAjOZX0PBz6V5A56ra2XVtVX6XVdvq1b/mv0uoP36AHoWmZ/RK/1uZXeQKRnzfH3PL5b9x6t324//i69Dxbfoxd+l+9lfZfR2zdf6W7nz774j/028CdJbgf+uNvmsL2M3mGHDX2t3/4W67nAl+ntj2uB11TVVbBbK/qEPdaq/V6qvPC7Fr8kFwJHVdUZe114+Nt+IvCWqjpxrwuPmSR/CGyrqrcOYV1bgOdX1UfmXZg0Qe456gKkFrquvZ8APkevRbiG//66TOttL6V3QoOr6Y0oPhd4/0Jse9iqaq4tTkkzsOtYi9XB9I7T/oBeN+NrgQ8s0LYDvIped+pngCl63Z2S9kN2HUuS1JAtWkmSGjJoJUlqaKwGQx1xxBG1bNmyUZchSdI+ue66675TVUcOmjdWQbts2TI2b9486jIkSdonSb420zy7jiVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpobE6BaPGV5J9Wt7LL0pSjy1azUlV7XE78ZwrBk43ZCXpvxm0kiQ1ZNBKktSQQStJUkPzDtokxye5JslUkpuSvLSbfl6Sbya5vrs9af7lSpI0WYYx6vhO4GVV9ekkBwPXJflwN+/1VfXnQ9iGJEkTad5BW1Vbga3d/duTTAHHzne9kiQtBkM9RptkGfAw4FPdpLOTfDbJxUkOH+a2JEkLL8k+3TTEoE1yEPBe4Peq6vvAXwH3B06l1+J97QzPOyvJ5iSbt23bNqxyJEkN+J36fTeUM0MlWUIvZN9RVe8DqKpv981/G3DFoOdW1UXARQArVqzwrzJip7zqarbv2Dnn5ZetvXJOyx26dAk3nHva3S1LkibWvIM2vb6BDcBUVb2ub/rR3fFbgKcBN853W2pv+46dbLngyUNf71wDWZIWm2G0aB8NPBf4XJLru2mvBFYnORUoYAvwwiFsS5KkiTKMUcebgEFHvD8433VLkjTpvHqPdnPw8rU89NK1DdYLMPwuaUkadwatdnP71AUeo5UE7NvgSAdGzsyglSQN1GJw5P74oduLCkiS1JAtWu2hxSfOQ5cuGfo6JWkSGLTazb50Ey1be2WT47mStJjYdSxJUkMGrSRJDRm0kiQ1ZNBKktSQg6EkSQO1OFPc/niWOINWkjRQizPFecIKSZI0VAatJEkNGbSSJDVk0EqS1JCDoTQnSQZPv3Dw8lXVsBpJmhy2aDUnVfXj22WXXcZJJ53EPe5xD0466SQuu+yy3eYbspL032zRap9s3LiRdevWsWHDBlauXMmmTZtYs2YNAKtXrx5xdZI0fmzRap+sX7+eDRs2sGrVKpYsWcKqVavYsGED69evH3VpkjSWbNFqn0xNTbFy5crdpq1cuZKpqakRVSSppWGfYGJ/vDa1Qat9snz5cjZt2sSqVat+PG3Tpk0sX758hFVJamGuZ4Xy2tSzm3fXcZLjk1yTZCrJTUle2k2/d5IPJ/li9/Pw+ZerUVu3bh1r1qzhmmuuYefOnVxzzTWsWbOGdevWjbo0SRpLw2jR3gm8rKo+neRg4LokHwbOBD5aVRckWQusBc4ZwvY0QrsGPL3kJS9hamqK5cuXs379egdCSdIM5h20VbUV2Nrdvz3JFHAs8BTgsd1ilwIfx6BdFFavXm2wStIcDXXUcZJlwMOATwH360J4Vxjfd5jbkiRpEgwtaJMcBLwX+L2q+v4+PO+sJJuTbN62bduwypEkaSwMZdRxkiX0QvYdVfW+bvK3kxxdVVuTHA3cOui5VXURcBHAihUrPKWQJI0xT8e674Yx6jjABmCqql7XN+ty4Izu/hnAB+a7LUnSaE0/3erebhpOi/bRwHOBzyW5vpv2SuAC4N1J1gBfB54xhG1JkjRRhjHqeBMwuC8Bfmm+65ckaZJ5rmNJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpoWFc+F0aK8lMl0cerKoaVSJJtmi1CFXVHrcTz7li4HRDVlJrBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ359R5NtFNedTXbd+yc07LL1l45p+UOXbqEG849bT5lSdKPDSVok1wM/Apwa1Wd3E07D3gBsK1b7JVV9cFhbE/aZfuOnWy54MlDXedcA1mS5mJYXceXAKcPmP76qjq1uxmykqT9zlCCtqo+AXx3GOuSJGkxaT0Y6uwkn01ycZLDG29LkqSx0zJo/wq4P3AqsBV47aCFkpyVZHOSzdu2bRu0iCRJEyvDOtdrkmXAFbsGQ811Xr8VK1bU5s2bh1KP9g8PvfShTdb7uTM+12S9khanJNdV1YpB85p9vSfJ0VW1tXv4NODGVtvS/uv2qQscdSxprA3r6z0bgccCRyS5GTgXeGySU4ECtgAvHMa2JEmaJEMJ2qpaPWDyhmGsW5KkSeYpGCVJashTMGriDfuY6qFLlwx1fZL2bwatJtpcB0ItW3vl0AdNSdJc2HUsSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDnhlKi06SwdMvHLz8sK7JLEmDGLRadAxOSePErmNJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoyaCVJasiglSSpIYNWkqSGhhK0SS5OcmuSG/um3TvJh5N8sft5+DC2JUnSJBlWi/YS4PRp09YCH62qBwIf7R5LkrRfGUrQVtUngO9Om/wU4NLu/qXAU4exLUmSJknLY7T3q6qtAN3P+zbcliRJY2nkg6GSnJVkc5LN27ZtG3U5kiQNVcug/XaSowG6n7cOWqiqLqqqFVW14sgjj2xYjiRJC69l0F4OnNHdPwP4QMNtSZI0lob19Z6NwCeBn01yc5I1wAXAE5J8EXhC91iSpP3KUC78XlWrZ5j1S8NYvyRJk2rkg6EkSVrMDFpJkhoyaCVJasiglSSpIYNWkqSGDFpJkhoaytd7JEl3X5J9Wr6qGlWiFmzRStKIVdUetxPPuWLgdEN28hi0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDXlmKElaIKe86mq279g55+WXrb1yTssdunQJN5x72t0tS40ZtJK0QLbv2MmWC5489PXONZA1GnYdS5LUkEErSVJDBq0kSQ0ZtJIkNeRgKEnSojQu1/lt3qJNsiXJ55Jcn2Rz6+1JkgSDr/M727V+W1moFu2qqvrOAm1LksbSwcvX8tBL1zZYL8Dwvzak4bDrWJIWyO1TF/g92v3QQgyGKuDqJNclOWsBtidJ0thYiBbto6vqW0nuC3w4yb9X1Sd2zezC9yyAE044YQHKkSRp4TRv0VbVt7qftwLvBx4xbf5FVbWiqlYceeSRrcuRJGlBNQ3aJD+V5OBd94HTgBtbblOSpHHSuuv4fsD7u+8y3RO4rKquarxNSZLGRtOgraqvAKe03IYkSS0uQTisyw/69R5JWkAtvopz6NIlQ1/npGlxCcJh/a0MWklaIPsSBMvWXtnkO7eLVYuTgQzrRCAGrSRp4rU4GciwWrRevUeSpIYMWkmSGjJoJUlqyGO0kqRFYdgjuoc1mtuglSRNvHEe0W3XsSRJDRm0kiQ1ZNBKktSQQStJUkMGrSRJDTnqWJJGrLuU6J7TLxy8fFU1rGbxmGm/wuB922q/GrSSNGIGZxvjsl/tOpYkqaFF1aKdrZtgkHH5tCNJWrwWVYu2qgbeTjznioHTJUlqbVEFrSRJ48aglSSpIYNWkqSGJnYw1CmvuprtO3bOefm5Xj7p0KVLuOHc0+5uWZIk7aZ50CY5Hfg/wAHAX1fVBcNY7/YdO5tc5mjY1zOUJO3fmnYdJzkAeDPwROAhwOokD2m5TUmSxknrY7SPAL5UVV+pqv8LvBN4SuNtSpI0Nlp3HR8LfKPv8c3AI2da+LbbbuOSSy6Z04pP/4nbuOSSbfMqbiHXK026a6+9dp+Wf8xjHtOoEmmytA7aQadq2u1MEUnOAs4COPbYYxuXI+numik4/+Urt/Gon7nPAlcjTY60PENSkl8AzquqX+4evwKgqv500PIrVqyozZs3z2ndy9Ze2WwwVIv1SouV/zMSJLmuqlYMmte6RftvwAOT/DTwTeBZwLOHseKDl6/loZeuHcaqpq0XwDcNSdJwNA3aqrozydnAh+h9vefiqrppGOu+feoCv94jSRp7zb9HW1UfBD7YejuSJI2jiT0zlKR2PPOaNDwTHbQtunkPXbpk6OuUJo1nXpOGZ2KDdl/eBBwVKUkaFa/eI0lSQxPbopUWg2TQOV1m1vJ775LasEUrjVBVDbydeM4VA6dLmjwGrSRJDRm0kiQ1ZNBKktTQohoMNdvAkly45zSPeUmSWltUQWtwSsPhRTuk4VlUQStpOLxohzQ8HqOVJKkhg1aSpIbsOpY0kBftkIbDoJW0By/aIQ2PXceSJDVk0EqS1JBBK0lSQwatJEkNGbSSJDVk0Epa9DZu3MjJJ5/MAQccwMknn8zGjRtHXZL2I369R9KitnHjRtatW8eGDRtYuXIlmzZtYs2aNQCsXr16xNVpf9CsRZvkvCTfTHJ9d3tSq21J0kzWr1/Phg0bWLVqFUuWLGHVqlVs2LCB9evXj7o07Sdat2hfX1V/3ngbkjSjqakpVq5cudu0lStXMjU1NaKKtL+x61haIKe86mq279g55+XncgrEQ5cu4YZzT5tPWYve8uXL2bRpE6tWrfrxtE2bNrF8+fIRVqX9SeugPTvJbwKbgZdV1fcab08aW9t37Bz6qQq97NzerVu3jjVr1uxxjNauYy2UeQVtko8ARw2YtQ74K+DVQHU/Xwv81oB1nAWcBXDCCSfMpxxJ2sOuAU8veclLmJqaYvny5axfv96BUFow8wraqnr8XJZL8jbgihnWcRFwEcCKFStqPvVI0iCrV682WDUyLUcdH9338GnAja22JUnSuGp5jPbPkpxKr+t4C/DChtuSJGksNQvaqnpuq3VLkjQp/HqPpDlJMvO8C/ecVuWQCwkMWklzZHBKd49BKy2Qg5ev5aGXrh3yOgGG+91cScNl0EoL5PapCzxhhbQf8jJ5kiQ1ZNBKktSQQStJUkMeo5W0KM32daTpHFGtlgxaaQENe/DSoUuXDHV9i8mg8Fy29sqhD0iT9saglRbIvrzBGwjS4uExWkmSGrJFK2minfKqq9m+Y+ecl59r9/2hS5dww7mn3d2ypB8zaEfIwRrS/G3fsbNJN7snA9Gw2HU8QlW1x+3Ec64YOF2SNJkMWkmSGrLrWNJEa3Gxht56wQs2aBgMWkkTrcXFGsBjtBoeu44lSWrIoJUkqSGDVpKkhgxaSZIacjCUpInXYuCSF2zQsMwraJM8AzgPWA48oqo29817BbAGuAv43ar60Hy2JUmDeLEGjbv5tmhvBH4deGv/xCQPAZ4FnAQcA3wkyYOq6q55bk9aVGY7DWcu3HOaZwmTJs+8graqpmDgm8VTgHdW1Y+Aryb5EvAI4JPz2d6k8qTnmonBKS1+rY7RHgv8S9/jm7tp+yVPei5J+6+9Bm2SjwBHDZi1rqo+MNPTBkwb+NE9yVnAWQAnnHDC3sqRJGmi7DVoq+rxd2O9NwPH9z0+DvjWDOu/CLgIYMWKFfajSZIWlVbfo70ceFaSA5P8NPBA4F8bbUuSpLE1r6BN8rQkNwO/AFyZ5EMAVXUT8G7g88BVwO844liStD+a76jj9wPvn2HeemD9fNa/WHgZL0naf3lmqAXgZbwkaf/luY4lSWrIFu0C8Vys0sKa6axbnnFLC82gXQCei1VaeIanxoVdx5IkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNeSZoUbIU8RJ0uJn0I6Q4SlJi59dx5IkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDBq0kSQ0ZtJIkNWTQSpLUkEErSVJDGafz7SbZBnytwaqPAL7TYL0tWGsbk1QrTFa91tqGtbbTot4Tq+rIQTPGKmhbSbK5qlaMuo65sNY2JqlWmKx6rbUNa21noeu161iSpIYMWkmSGtpfgvaiURewD6y1jUmqFSarXmttw1rbWdB694tjtJIkjcr+0qKVJGkkJj5okzwtSSV5cN+0Bya5IsmXk1yX5Jokv9jNOzPJtiTX990esoD1rktyU5LPdtt+ZJKPJ/mPvnre0y37F0n+aNpz37yAtd7V1XNDkk8n+R/d9GXdPn9137JHJNmZ5E3d4/OS/P5C1boPde/o5n0+yVuSjPx/YIbXxNlJvtTt5yNGXeMuM9T6ju71e2OSi5MsGXWdMGOtG7rXxWeTvCfJQaOuc5dB9fbNe2OSO0ZZX78Z9u0lSb7a9z526qjrhBlrTZL1Sb6QZCrJ7zYtoqom+ga8G/hH4Lzu8U8CXwB+rW+Zk4Ezu/tnAm8aUa2/AHwSOLB7fARwDPBxYMWA5Q8BvgL8DPDTwFeBwxaw3jv67v8ycG13fxnwZeAzffNfDFy/a98C5wG/P6L9PFvdN3b37wl8Avj1UdQ4h9fEw7p6twBHjLLGOdT6JCDdbSPw4jGu9ZC+ZV4HrB11rbPV291fAfxt/+t6HGsFLgGePur65ljr84C3A/fopt+3ZR33ZIJ1n0YfDawCLqf35v4c4JNVdfmu5arqRuDGUdQ4zdHAd6rqRwBV9R2AJAMXrqrvJ1kHvKmb9MdV9Z8LUegAhwDf63u8A5hKsqKqNgPPpPeh55hRFDeL6XUDUFV3Jvln4AELX9JuBr4mgG/BzK+NEZm1VoAk/wocN4LappupVgDS27FLgXEZpDLTe8MBwGuAZwNPG115u9mn97ERm6nWFwPPrqr/102/tWURI+82m6enAldV1ReA7yb5OeAk4NN7ed4zp3UdL21eac/VwPFdd8VfJnlM37x39NXzml0Tq2ojcDi9T+J/u0B17rK0q+ffgb8GXj1t/juBZyU5DriLvjfcEdtb3SS5F/BLwOcWurhpZntNjJtZa+26jJ8LXDWS6nY3Y61J/ga4BXgw8MZRFTjNTPWeDVxeVVtHWNt0s70O1nddtK9PcuCoCuwzU633p5cDm5P8Q5IHtixi0oN2Nb03e7qfq6cvkOT93bGj9/VNfldVndp327EQxVbVHcDPA2cB24B3JTmzm/2cvnr+oK/+44CjgGNGcDxpR1fPg4HTgbdn94+tVwFPoLff37XAtc1mtrrvn+R64J+AK6vqH0ZWJXt9TYyVOdT6l8AnquofR1DebmartaqeR6/nZYpeT8zIzVDvK4FnMD4fBoBZ9+0r6H14eThwb+CcUdW4yyy1Hgj8V/XODvU24OLWhUzkDbgPve7Lr9E7jvUN4OvAGuDSacuuAD7e3T+TER2jHfA7PB34e2Y4Rtst817gDOAC4DULXN8d0x5/G7gvux/rvJhe6+A+/fuWMTlGO1Pd43rb9Zroe7yFMTlGO1utwLnA39Ed8xq32/T92k17DHDFqGubpd6d3f/Wlu72/4Avjbq2Oe7bx47jvu17z/13YFk3LcD2ltud5Bbt04G3V9WJVbWsqo6nN1joC8Cjk/xa37L3GkmF0yT52WldFKcyy0UUkjyRXkC8nV7359OygCOkp9XyYOAA4LZps14LnFNV06ePhVnqHgv7+poYpZlqTfJ8eoPOVld3zGvUZqj160ke0M0P8Kv03nBHboZ631pVR3Xvb8uAH1bVqMcUzPY6OLqbH3qH9UY+LmaW/6+/Ax7XTXsMvdxoZpIHQ62m18rr9156gwZ+BXhdkjfQa83cDpzft9wzk6zse/zbVfXPLYvtHAS8MclhwJ3Al+h1abyH3jHaXV3Y36H3O7yB3ii+An6Q5OX0BkY9bo81t7G062aF3qe+M6rqrv7e46q6CbhpgeqZq73WPUYGvia6rxu8nN5hg88m+WBVPX+EdcLMr99b6L15fbLbx++rqj8ZWZU9g2p9EfD+JIfQe13cQG+0/DiYad+Oo5lqfXeSI+nt2+vp7e9Rm6nWO+m95/5P4A6g6f+WZ4aSJKmhSe46liRp7Bm0kiQ1ZNBKktSQQStJUkMGrSRJDRm0kiQ1ZNBKktSQQStJUkP/HyZcyzdwfQPxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "rng = np.random.RandomState(1) # make sure the results are repeatable\n",
    "feature_names = nX.columns.tolist() # write the column names to a list\n",
    "# cross_validate takes the particular model, in this case ridge regularization\n",
    "# and undertakes a number of runs according the method specified by cv=\n",
    "# RepeatedKFold splits the data into n sections and repeat the regression modelling 5 times, \n",
    "# giving 25 runs\n",
    "# return_estimator=True returns the fitting data for each run\n",
    "scores = cross_validate(Ridge(alpha=20.), nX, Y, \n",
    "                        cv=RepeatedKFold(n_splits=5, n_repeats=5), return_estimator=True)\n",
    "# take the results for each simulation (estimator), extract the coefficients for each run \n",
    "# and add them to a dataframe with columns being the feature names\n",
    "coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)\n",
    "# plot the descriptive statics of the coefficients in a box and whisker plot to show \n",
    "# variability\n",
    "ax = coefs.plot(kind='box',figsize=(10,5))\n",
    "plt.title('Ridge regression, alpha = 20.')\n",
    "plt.axhline(y=0, color='.5')\n",
    "plt.subplots_adjust(left=.3)\n",
    "print('Testing score is',\n",
    "      round(np.mean(scores['test_score']),4),\"±\",round(np.std(scores['test_score']),4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use `RidgeCV` to estimate the optimal value of `alpha` using the array defined above. You should also investigate the\n",
1051
    "effect of changing the number of cross-validations in the range [4,10]. In general, you should get a similar optimal value of `alpha` to the example above. The use of cross-validations, where the testing and training sets are changed, creates a more robust method than the graphical method above. (4 marks)"
Simon Clarke's avatar
Simon Clarke committed
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import RidgeCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two possible ways to approach this. The first is to use the training sets to train the model, and the testing sets to test the model. In this case you want to calculate the score using `ridge.score`. This gives an optimal `alpha` of approximately 25."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cv= 3 Testing score = 0.4626 Optimal alpha =  25.12\n",
      "cv= 4 Testing score = 0.4629 Optimal alpha =  15.85\n",
      "cv= 5 Testing score = 0.4626 Optimal alpha =  25.12\n",
      "cv= 6 Testing score = 0.4626 Optimal alpha =  25.12\n",
      "cv= 7 Testing score = 0.4626 Optimal alpha =  25.12\n",
      "cv= 8 Testing score = 0.4574 Optimal alpha =  39.81\n",
      "cv= 9 Testing score = 0.4626 Optimal alpha =  25.12\n",
      "cv= 10 Testing score = 0.4203 Optimal alpha =  100.0\n"
     ]
    }
   ],
   "source": [
    "X_train, X_test, Y_train, Y_test = train_test_split(nX, Y, test_size=0.8) \n",
    "for CV in range(3,11):\n",
    "    ridge = RidgeCV(alphas=alfas,cv=CV).fit(X_train, Y_train)\n",
    "    scores = ridge.score(X_test, Y_test)\n",
    "    alfa_opt = ridge.alpha_\n",
    "    print('cv=',CV,'Testing score =',round(scores,4),'Optimal alpha = ',round(alfa_opt,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second way is to use `nX` and `Y`, and the cross validation routine will return the best score and the best `alpha` as `ridge.best_score_` and `ridge.alpha_`, respectively. This gives an optimal `alpha` of approximately 40.\n",
    "\n",
    "If you use `cv=RepeatedKFold(n_splits=CV, n_repeats=5)`, then results are consistent with method above, but take longer to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cv= 3 Testing score = 0.4313 Optimal alpha =  25.12\n",
      "cv= 4 Testing score = 0.4957 Optimal alpha =  15.85\n",
      "cv= 5 Testing score = 0.3883 Optimal alpha =  25.12\n",
      "cv= 6 Testing score = 0.4279 Optimal alpha =  25.12\n",
      "cv= 7 Testing score = 0.4215 Optimal alpha =  25.12\n",
      "cv= 8 Testing score = 0.1691 Optimal alpha =  39.81\n",
      "cv= 9 Testing score = 0.399 Optimal alpha =  25.12\n",
      "cv= 10 Testing score = -0.206 Optimal alpha =  100.0\n"
     ]
    }
   ],
   "source": [
    "for CV in range(3,11):\n",
    "    ridge = RidgeCV(alphas=alfas,cv=CV).fit(X_train, Y_train)\n",
    "    scores = ridge.best_score_\n",
    "    alfa_opt = ridge.alpha_\n",
    "    print('cv=',CV,'Testing score =',round(scores,4),'Optimal alpha = ',round(alfa_opt,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
1140
    "For the optimal value of `alpha` found using `RidgeCV` in the previous analysis, rerun the analysis of the coefficients and calculation of the model accuracy, but in this case also investigate the effect of individually dropping the columns `AGE`, `S2` and `S4`. (6 marks)\n",
Simon Clarke's avatar
Simon Clarke committed
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
    "\n",
    "Since we are repeating the process, first define a convenience function (don't have to do this, but it makes debugging easier)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ridge_coefs_analyse(Xc, Yc, alfac, nsplits, nrepeats):\n",
    "    '''Analyse the coefficients for ridge regression with cross validation'''\n",
    "    feature_names = Xc.columns.tolist() # write the column names to a list\n",
    "    scores = cross_validate(Ridge(alpha=alfac), Xc, Yc, \n",
    "                            cv=RepeatedKFold(n_splits=nsplits, n_repeats=nrepeats), return_estimator=True)\n",
    "    coefs = pd.DataFrame([est.coef_ for est in scores['estimator']],columns=feature_names)\n",
    "    ax = coefs.plot(kind='box',figsize=(10,5))\n",
    "    plt.title('Ridge regression, alpha = %5.2f ' % alfac)\n",
    "    plt.axhline(y=0, color='.5')\n",
    "    plt.subplots_adjust(left=.3)\n",
    "    print('Testing score is',\n",
    "          round(np.mean(scores['test_score']),4),\"±\",round(np.std(scores['test_score']),4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop the `AGE` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing score is 0.4811 ± 0.0557\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAf00lEQVR4nO3de5QdZZnv8d/P0GocQkAT7oRWLtIQDlFb9Bz7iFG5O6M4Otp6GMTGgCMZZpYXWnscQOg5QUVcE3AknM4EXNDqAlEkiCC0aM/xMh0FTNxeEMOIBAiIIdHA6cBz/qhq3Gn37tve777197NWrd5Vb9X7PvvS+9n1VtVbjggBAIA0nlPvAAAAaGUkWgAAEiLRAgCQEIkWAICESLQAACREogUAICESLerO9udtf3yC8rB9cC1jalSTvVY1iuF1th+o9rpAqyLRIjnbG21vt73N9kO219jedaw8Is6KiAvrGWOz4LWaPtsn2x62/fv883el7XlF5Wts/7/88zk2zZmgvn/M69lie7Xt5xWVtdsesv1H2z+z/cbUzw+Nj0SLWvnLiNhV0hJJL5P00TrHsxNnqvb/UO36UJH5ki6StK+kDkn7S/rUuHU+GRG7Fk1Pl6rI9vGSeiW9QVK7pJdIuqBolUFJP5b0Ikl9kq6zvbCKzwVNiC8C1FREPCTpm8oSrqRn9yguKpr/sO1Nth+0/d7i7W2/yPbXbT9h+z9tX2R7uKj8MNu32f6d7Z/b/ptysdj+tu1+2/8h6Y+SXmJ7vu2BvP3f5vXPydefY/sS24/a/rXts/Nu7V1mWN/Btu/M94wetf2lfLltX2r7kbzsHtuLy7xW77N9b/58b7S9b1FZ2D7L9i9tP277ctueyvtk+3TbBdtbbd9n+8wJ1t1o+6O2f5q38++2nz9unQ/mz2eT7dOLlp9s+8f5+/kb2+dPJb7piIhrI+KWiPhjRDwu6UpJr5lhdadJGoiIDXldF0p6jyTZPlTSyyWdFxHbI+J6ST+R9NcVPwk0NRItasr2/pJOlHRvmfITJH1I0rGSDpE0vuvtckl/kLS3si+904q2/QtJt0m6VtKekrolfc72EROEdKqkZZLmSbpf0lWSdkg6WNme93GSzsjXfV8e+xJlX6hvqbC+CyXdKmkPZXtZK/Plx0l6raRDJe0u6R2SHhvfkO3XS/rfkv5G0j55e18ct9qbJL1S0lH5esdP8FoUeyTfdjdJp0u61PbLJ1j/3XndB+Vx/1NR2d7K9ir3k9Qj6XLbe+Rlf5D0t/nzPFnS+22Xel1le5Gz7t9y07um+NxeK2nDuGV/l/9YWWd7osR4hKS7i+bvlrSX7RflZfdFxNZx5RN9/jAbRAQTU9JJ0kZJ2yRtlRSSbpe0e1H5GkkX5Y9XS1pRVHZovs3BkuZIGpX00qLyiyQN54/fIem749q+QtkeRqm4vi3pE0Xze0l6StLcomXdkobyx3dIOrOo7I15bLvMsL6rJa2StP+4uF4v6ReSXi3pOePKil+rAWVdnmNlu+avT3s+H5K6isq/LKl3hu/hVyWdkz9+naQHxr2/ZxXNnyTpV0Xrbh97jfJlj0h6dZl2Pivp0oSfxWMlPS7p0KJlL1fW1btLHvtWSa8ps/2vJJ1QNN+Wv87tyn5kfX/c+v2S1qT+H2Nq7Ik9WtTKWyJinrIv3sMkLSiz3r6SflM0f3/R44XKvgyLy4sfHyjpVcV7Ocr2tPaeIK7x27dJ2lS0/RXK9o5LxVb8eCb1fUSSJf3Q9oaxbvKIuEPSZcr23h+2vcr2biXa2ldFr09EbFO257tf0ToPFT3+o7JkPCnbJ9r+fr6X93tlCajceyb9+Xu2b9H8YxGxo1Qctl/l7OShzba3SDprknZmzParlfV2vC0ifjG2PCJ+FBGPRcSOiLhZ0jWS3lqmmm3K9vLHjD3eWqJsrHyrMKuRaFFTEXGnsr2yT5dZZZOkA4rmFxU93qysG3b/omXF6/5G0p0RsXvRtGtEvH+ikMZt/5SkBUXb7xYRY11/myZoe9r1RcRDEfG+iNhX0pnKurkPzsv+NSJeoazb8VBJHy7R1oPKkrmkZ7vOXyTptxM830k5O4v2emXv0V4Rsbukm5X9KChn/Hv24BSbu1bSjZIOiIj5kj5frp2863jbBNO7J3hOL8vbeW9E3D5JTFEuBmVdzkcVzR8l6eGIeCwve4mLzmjOy8d3U2OWIdGiHj4r6VjbS0qUfVnSe2wfbvsFks4bK4jsTNCvSDrf9gtsH6bs+N6YmyQdavtU22359ErbHVMJKiI2KTtmeont3Ww/x/ZBto8piu0c2/vZ3l3SuZXUZ/vt+TFrKevODElP5zG/ynabsmOYT0oqdRbstZJOt70kT47/IukHEbFxsufq7DKUsN1eovi5kp6n/IeN7ROVHTeeyAds72/7hZI+JulLk8WQmyfpdxHxpO2jJZU9zhoR/xU7nxk8frqm1Hb5iWS3SFoeEV8vUf4227vm789xkv6XsqRcytWSevLP5x7KjkWvyeP7haS7JJ1n+/m2T5H035T9aMEsRqJFzUXEZmVfWH828EJEfENZIr5D2QlTd4xb5WxlJ9Y8JOkLyi6neCrfdquyhPBOZXtUD0m6WFnSmKq/VZZofqos+V2n7EQjKTtb9VZJ9yi7hONmZXvYJS8FmUJ9r5T0A9vblH2xnxMRv1bW3Xhlvv79yrqD/6wHIN8z+7iyL/JNyk5EeucUn+cBed1/tvebv45/r+yHxePKkl+5xDPmWmWvzX35dNHEqz/r7yR9wvZWSf+ct1ltH1R22GGgaO+3eC/zHGWvw++VXfbzvoj4trTTXvQiSYqIWyR9UtKQstfvfhX9GFT2+ncqe91WKOum3pzgOaGJOIIbv6N52b5Y0t4RcdqkK1e/7RMlfT4iDpx05QZj+58kbY6IK6pQ10ZJZ0TEtyoODGhBu9Q7AGA68u7i5yq7PvGVyi4XOWPCjarX9lxJS5Xtue2lbE/mhlq0XW0RMdU9TgAVousYzWaesuO0f1DWzXiJpK/VqG0rGwXocWVdxwVl3Z0AUBZdxwAAJMQeLQAACZFoAQBIqKFOhlqwYEG0t7fXOwwAAKZl3bp1j0ZEyTs1NVSibW9v18jISL3DAABgWmzfX66MrmMAABIi0QIAkBCJFgCAhEi0AAAkRKIFACAhEi0AAAmRaAEASIhECwBAQiRaAAASItECAJBQQw3BiIztaW/D7Q4BoDGxR9uAIqLkdOC5N5UtAwA0JhItAAAJkWgBAEio4kRr+wDbQ7YLtjfYPidffr7t39q+K59OqjxcAEAtDQ4OavHixZozZ44WL16swcHBeofUdKpxMtQOSR+MiB/Znidpne3b8rJLI+LTVWgDAFBjg4OD6uvr08DAgLq6ujQ8PKyenh5JUnd3d52jax4V79FGxKaI+FH+eKukgqT9Kq0XAFBf/f39GhgY0NKlS9XW1qalS5dqYGBA/f399Q6tqVT1GK3tdkkvk/SDfNHZtu+xvdr2HmW2WWZ7xPbI5s2bqxkOAKAChUJBXV1dOy3r6upSoVCoU0TNqWqJ1vaukq6X9A8R8YSkf5N0kKQlkjZJuqTUdhGxKiI6I6Jz4cKF1QoHAFChjo4ODQ8P77RseHhYHR0ddYqoOVUl0dpuU5Zkr4mIr0hSRDwcEU9HxDOSrpR0dDXaAgDURl9fn3p6ejQ0NKTR0VENDQ2pp6dHfX199Q6tqVR8MpSzYYwGJBUi4jNFy/eJiE357CmS1lfaFgCgdsZOeFq+fLkKhYI6OjrU39/PiVDT5EpHFbLdJem7kn4i6Zl88cckdSvrNg5JGyWdWZR4S+rs7IyRkZGK4mkmR11wq7ZsH01W//y5bbr7vOOS1Q8AyNheFxGdpcoq3qONiGFJpQbnvbnSulvdlu2j2rji5GT1t/euTVY3gNbEWOvVx8hQAIBnMdZ69XH3njqa19GrI6/qTVi/JKXbYwYATI5EW0dbCyvoOgaAFkfXMQAACZFoAQBIiK7jOkvZvTt/bluyugEAU0OiraPpHp9t712b9JguAKD66DoGACAh9mgBYBaaych00znUxch0f0KiBYBZiJHpaoeuYwAAEmKPFgBmIUamqx0SLQDMQoxMVzt0HQMAkBCJFgCAhEi0AAAkxDHaBjTRjZd9cenl3BMSwHQxBGxtkGgbEEkTQGoMAVs7dB0DAJAQiRYAgIRItE1gcHBQixcv1pw5c7R48WINDg7WOyQAwBRVnGhtH2B7yHbB9gbb5+TLX2j7Ntu/zP/uUXm4s8/g4KD6+vq0cuVKPfnkk1q5cqX6+vpItgDQJKqxR7tD0gcjokPSqyV9wPbhknol3R4Rh0i6PZ/HNPX392tgYEBLly5VW1ubli5dqoGBAfX399c7NADAFFScaCNiU0T8KH+8VVJB0n6S3izpqny1qyS9pdK2ZqNCoaCurq6dlnV1dalQKNQpIgDAdFT1GK3tdkkvk/QDSXtFxCYpS8aS9qxmW7NFR0eHhoeHd1o2PDysjo6OOkUEAJiOqiVa27tKul7SP0TEE9PYbpntEdsjmzdvrlY4LaOvr089PT0aGhrS6OiohoaG1NPTo76+vnqHBgCYgqokWtttypLsNRHxlXzxw7b3ycv3kfRIqW0jYlVEdEZE58KFC6sRTkvp7u5Wf3+/li9fruc///lavny5+vv71d3dXe/QAMwCY1c93P/Jv+KqhxmqeGQoZ+MFDkgqRMRniopulHSapBX5369V2tZs1d3dTWIFUHNjVz0MDAzoPTc/oZUn7aaenh5J4jtpGqqxR/saSadKer3tu/LpJGUJ9ljbv5R0bD4PAGgSxVc9eM4uXPUwQ26kcXU7OztjZGSk3mEAwKw10U1NymmkPFIvttdFRGepMkaGAgA8KyKenY444gjdcccdOy274447dMQRR+y0DBMj0QIASuKqh+rgNnkAgJLGTnhavny5CoWCOjo6uOphBjhGCwBAhThGCwBAnZBoAQBIiEQLAEBCJFoAABIi0QIAkBCJFgCAhEi0AAAkRKIFACAhRoZC1TEoOQD8CXu0qLriwcaLpwPPvalsGQC0KhItAAAJkWgBAEiIRAsAQEIkWgAAEiLRAgCQEIkWAICESLQAACREogUAIKGqjAxle7WkN0l6JCIW58vOl/Q+SZvz1T4WETdXoz00hqMuuFVbto9Oa5v23rVTXnf+3Dbdfd5x0w0LABpKtYZgXCPpMklXj1t+aUR8ukptoMFs2T6qjStOTlb/dJIyADSqqnQdR8R3JP2uGnUBANBKUh+jPdv2PbZX294jcVsAADQcV2tAd9vtkm4qOka7l6RHJYWkCyXtExHvLbHdMknLJGnRokWvuP/++6sSD9I78qojk7fxk9N+krwNAKiU7XUR0VmqLNlt8iLi4aIArpR0U5n1VklaJUmdnZ3cxqWJbC2s4BgtAEwiWdex7X2KZk+RtD5VWwAANKpqXd4zKOl1khbYfkDSeZJeZ3uJsq7jjZLOrEZbAAA0k6ok2ojoLrF4oBp1AwDQzBgZCgCAhEi0AAAkRKIFACAhEi0AAAmRaAEASCjZgBWYHVIOKjF/bluyugGgVki0mLHpjgrV3rs26UhSANCI6DoGACAhEi0AAAmRaAEASIhECwBAQiRaAAASItECAJAQiRYAgIRItAAAJESiBQAgIRItAAAJMQQjqs52+bKLSy+PiETRAEB9kWhRdSRNAPgTuo4BAEiIRAsAQEIkWgAAEqpKorW92vYjttcXLXuh7dts/zL/u0c12gIAoJlUa492jaQTxi3rlXR7RBwi6fZ8HgCAWaUqiTYiviPpd+MWv1nSVfnjqyS9pRptAQDQTFIeo90rIjZJUv53z4RtAQDQkOp+MpTtZbZHbI9s3ry53uEAAFBVKRPtw7b3kaT87yOlVoqIVRHRGRGdCxcuTBgOAKRne9oTWlvKRHujpNPyx6dJ+lrCtgCgIUREyenAc28qW4bWVq3LewYlfU/SS20/YLtH0gpJx9r+paRj83kAAGaVqox1HBHdZYreUI36AQBoVnU/GQoAgFZGogUAICESLQAACZFoAQBIiEQLAEBCJFoAABIi0QIAkBCJFgCAhEi0AAAkRKIFACAhEi0AAAmRaAEASKgqNxUAAKARzOT+vqlvVcgeLQCgZTTi/YBJtAAAJETXMQDMwFEX3Kot20entU1779oprzt/bpvuPu+46YaFBkSiBYAZ2LJ9VBtXnJys/ukkZTQ2uo4BAEiIRAsAQEIkWgAAEiLRAgCQEIkWAICEkp91bHujpK2Snpa0IyI6U7cJAKnN6+jVkVf1JqxfktKd1YzaqdXlPUsj4tEatQUAyW0trODyHkwJ19ECAJpOMw0YUotEG5JutR2SroiIVTVoEwDQwpppwJBaJNrXRMSDtveUdJvtn0XEd8YKbS+TtEySFi1aVINwAAConeRnHUfEg/nfRyTdIOnoceWrIqIzIjoXLlyYOhwAAGoqaaK1/Re25409lnScpPUp2wQAoJGk7jreS9IN+Y14d5F0bUTckrhNAAAaRtJEGxH3SToqZRsAADQyRoYCACAhrqMFgBlKOajE/LltyepuBc00MheJFgBmYLrXcLb3rk163eds00wjc9F1DABAQiRaAAASItECAJAQx2gBAE2pWU5GI9ECAJpOM52MRtcxAAAJkWgBAEiIRAsAQEIkWgAAEiLRAgCQEIkWAICESLQAACREogUAICESLQAACZFoAQBIiEQLAEBCjHUMAFVku3zZxaWXR0SiaNAISLQAUEUkTYxH1zEAAAklT7S2T7D9c9v32u5N3R4AAI0kaaK1PUfS5ZJOlHS4pG7bh6dsEwCARpL6GO3Rku6NiPskyfYXJb1Z0k9LrfzYY49pzZo1iUMCAMw2Jzz3Ma1Zs7kubadOtPtJ+k3R/AOSXlW8gu1lkpZJ0n777Zc4HABAK7vzzjvLl/2m9PJjjjkmUTQZpzxDzvbbJR0fEWfk86dKOjoilpdav7OzM0ZGRpLFA8wGE11eUg5nygKVsb0uIjpLlaU+GeoBSQcUze8v6cHEbQKzWkSUnA4896ayZQDSSZ1o/1PSIbZfbPu5kt4p6cbEbQIA0DCSHqONiB22z5b0TUlzJK2OiA0p2wQAoJEkHxkqIm6WdHPqdgAAaESMDAUAQEIkWgAAEiLRAgCQEIkWAICESLQAACTE/WiBJnXUBbdqy/bRaW3T3rt2yuvOn9umu887brphARiHRAs0qS3bR7VxxcnJ6p9OUgZQHokWGIexggFUE8dogXEYKxhANZFoAQBIiEQLAEBCJFoAABIi0QIAkBBnHQNNal5Hr468qjdh/ZKU7vIhYLYg0QJNamthBdfRAk2ArmMAABJijxZoYin3OufPbUtWNzCbkGiBJjXdbuP23rVJu5oBlEbXMQAACbFHi1mLu98AqAUSLWYt7n4DoBaSdR3bPt/2b23flU8npWoLAIBGlXqP9tKI+HTiNgAAaFh0HQMtZqL76fri0su51R+QTuqzjs+2fY/t1bb3SNwWAJW/n+5EE4B0Kkq0tr9le32J6c2S/k3SQZKWSNok6ZIydSyzPWJ7ZPPmzZWEAwBAw6mo6zgi3jiV9WxfKemmMnWskrRKkjo7O/lpDQBoKcmO0dreJyI25bOnSFqfqi1gJrj7DYBaSHky1CdtL5EUkjZKOjNhW8C0cfcbALWQLNFGxKmp6gYAoFkw1jEAAAmRaAEASIhECwBAQowMhVmNG6cDSI1Ei1lrttw4fXBwUP39/SoUCuro6FBfX5+6u7vrHVZZEw0hWQ6jW6GR0XUMtLDBwUH19fVp5cqVevLJJ7Vy5Ur19fVpcHCw3qGVVW6YyAPPvYkhJNGUSLRAC+vv79fAwICWLl2qtrY2LV26VAMDA+rv7693aMCsQdcxME4r3f2mUCioq6trp2VdXV0qFAp1igiYfdijBcZppbvfdHR0aHh4eKdlw8PD6ujoqFNEwOxDogVaWF9fn3p6ejQ0NKTR0VENDQ2pp6dHfX199Q4NmDXoOgZa2NjZxcuXL3/2rOP+/v6GPusYaDVupG6vzs7OGBkZqXcYAGrgqAtu1Zbto8nqnz+3TXefd1yy+oFittdFRGepspbco+U6PKDxbdk+yt2TMCu05DFarsMDADSKlky0AAA0ChItAAAJkWgBAEiIRAsAQEIkWgAAEmrJy3sANL55Hb068qrehPVLUvPd1hCtp6kT7UwueJ/OtXVc8A6ks7WwgutoMSs0daLlgncAQKOr6Bit7bfb3mD7Gdud48o+avte2z+3fXxlYQIA0Jwq3aNdL+mtkq4oXmj7cEnvlHSEpH0lfcv2oRHxdIXtAQDQVCrao42IQkT8vETRmyV9MSKeiohfS7pX0tGVtAUAQDNKdYx2P0nfL5p/IF8GAM9KeR7E/LltyeoGpmPSRGv7W5L2LlHUFxFfK7dZiWUlR+63vUzSMklatGjRZOEAaBHTPZGxvXdt0pMfgVQmTbQR8cYZ1PuApAOK5veX9GCZ+ldJWiVl96OdQVsAADSsVF3HN0q61vZnlJ0MdYikH1a7ES54BwA0uooSre1TJK2UtFDSWtt3RcTxEbHB9pcl/VTSDkkfSHHGMRe8AwAaXUWJNiJukHRDmbJ+Sf2V1A8AQLPjpgIAACREogUAICESLQAACZFoAQBIqKnv3iMxsgwAoLE1daJlZBkAQKNr6kQLoPXYpUZwzcsuLr08gkHl0LhItAAaCkkTrYaToQAASIhECwBAQiRaAAASItECAJAQiRYAgIRItAAAJESiBQAgIRItAAAJkWgBAEiIRAsAQEIkWgAAEmrJsY4ZlBwA0ChaMtGSNAEAjYKuYwAAEqoo0dp+u+0Ntp+x3Vm0vN32dtt35dPnKw8VAIDmU2nX8XpJb5V0RYmyX0XEkgrrBwCgqVWUaCOiIE188hEAALNZymO0L7b9Y9t32v6fCdsBAKBhTbpHa/tbkvYuUdQXEV8rs9kmSYsi4jHbr5D0VdtHRMQTJepfJmmZJC1atGjqkQMA0AQmTbQR8cbpVhoRT0l6Kn+8zvavJB0qaaTEuqskrZKkzs5OrssBALSUJF3HthfanpM/fomkQyTdl6ItAAAaWaWX95xi+wFJ/13SWtvfzIteK+ke23dLuk7SWRHxu8pCBQCg+VR61vENkm4osfx6SddXUjcAAK2AkaEAAEjIjTQusO3Nku5P2MQCSY8mrD814q8v4q8v4q8v4p/YgRGxsFRBQyXa1GyPRETn5Gs2JuKvL+KvL+KvL+KfObqOAQBIiEQLAEBCsy3Rrqp3ABUi/voi/voi/voi/hmaVcdoAQCotdm2RwsAQE21VKK13ZffiP6e/Ibzr7L9bds/L7oJ/XX5uv9q++Pjtr28ftFLtp/OY7zb9o9s/498ebvtsH1h0boLbI/aviyfP9/2h+oV+3iTPJftedlPbX/edkN+Dst8ns62fW/+fiyod4wTKRP/Nfn/w3rbq2231TvOcsrEP5B/pu6xfZ3tXesdZzml4i8qW2l7Wz3jm0yZ13+N7V8XfZ827D3Hy8Rv2/22f2G7YPvvaxJMRLTEpGwYyO9Jel4+v0DSvpK+LamzxPq7KRt/+SWSXizp15J2r/Nz2Fb0+HhJd+aP2yX9StKPi8rfL+kuSZfl8+dL+lC934cpPpf1+eNdJH1H0lvrHe80Pk8vy5/DRkkL6h3nDOI/SZLzaVDS++sd6zTj361onc9I6q13rNOJP3/cKekLxf8jjTZN8PqvkfS2esdXQfynS7pa0nPy5XvWIp6KhmBsMPtIejSyOwcpIh6Vyt+UPiKesN0n6bJ80T9HxO9rEegU7Sbp8aL57ZIKtjsjYkTSOyR9WdmHp9GNfy6SpIjYYfv/Sjq49iFNquTnSdKDUvnPVQOZMH5Jsv1DSfvXIbapKBe/JMnZGzBXUqOeZFLu+2iOpE9JepekU+oX3qSm9X3agMrF/35J74qIZ/Llj9QimIbsspuhWyUdkHcJfM72MUVl1xR1dXxqbGFEDEraQ9mv5C/UOuAS5uYx/kzS/5F04bjyL0p6p+39JT2toi/NBjTZc5HtF0h6g6Sf1Dq4KZjo89QMJow/7zI+VdItdYlucmXjt/3vkh6SdJiklfUKcBLl4j9b0o0RsamOsU3FRJ+f/rw79lLbz6tXgJMoF/9Bkt5he8T2N2wfUotgWibRRsQ2Sa9QdhP5zZK+ZPs9efG7I2JJPn14bJs8Ye0tad8GOdazPY/xMEknSLraO/+EvEXSsZK6JX2pHgFOw0TP5SDbd0n6D0lrI+IbdYuyjEk+Tw1vCvF/TtJ3IuK7dQhvUhPFHxGnK+vJKSjr2Wk4ZeL/mKS3q3F/HDxrgtf/o8p+4LxS0gslnVuvGCcyQfzPk/RkZCNEXSlpda0CaslJ0tskfV1ljtHm61wv6TRJKyR9qgFi3jZu/mFJe2rn45qrlf2af5Gk96gJjtGWey7NNI19normN6qBj9FOFL+k8yR9VflxqmaYxr/++bJjJN1U79imEf9o/r+7MZ+ekXRvvWOr4PV/XZO9/l+X9DNJ7fkyS9pSi/ZbZo/W9kvHdQMs0QQ3KLB9orIv/quVdWueYvvwtFFOne3DJM2R9Ni4oksknRsR45c3rAmeS8Oa7uep0ZSL3/YZyk5O6478OFUjKhP/f9k+OC+3pL9U9sXZcMrEf0VE7B0R7RHRLumPEdGI5ydM9PnZJy+3pLdIWl+P+CYzwf/vVyW9Pl92jKRf1CKeVjoZaldJK23vLmmHpHuVdRtcp+wY7fZ8vUclvUnSZ5WdPReS/mD7I8pOjHr9n9VcO3PzLlUp+7V1WkQ8Xdx7HBEbJG2oR3DTNOlzaXAlP0/55QAfUXbI4R7bN0fEGXWMs5xy/w8PKfvC+V7+XnwlIj5RtyjLKxX/WZJusL2bss/U3crOvm9E5V7/ZlEu/i/bXqjs9b9L2XvSiMrFv0NZPvhHSdsk1eR/l5GhAABIqGW6jgEAaEQkWgAAEiLRAgCQEIkWAICESLQAACREogUAICESLQAACZFoAQBI6P8Dj3ceDskEe34AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Xn = nX.drop(columns=['AGE'])\n",
    "ridge_coefs_analyse(Xn, Y, 25., 5, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop the `S2` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing score is 0.4857 ± 0.0825\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df5QcVZ338c/HkIUYIKAECAwwyg8ZHB6iO6I+5KhRQRBXxUdXRh8JZDT4g8DuUSE66wJCng1RxCy6KmxigkdGOCIrSxBBGHRnV10nyo/gKCImGggQEDFAxCR+nz+qku0M3T3Tmb5d3TPv1zl1pqtuVd1v/5j+9r11q8oRIQAAkMbzig4AAIDxjEQLAEBCJFoAABIi0QIAkBCJFgCAhEi0AAAkRKJF4Wx/2fanqpSH7cMaGVOzGum1alAMr7O9rt7rAuMViRbJ2V5je5Ptp2w/bHu57d23lUfEByPioiJjbBW8VrWzfbLtAdt/yD9/V9reo6R8ue0/55/PbdOkKvv7+3w/T9peZnvXkrJ22/22n7H9C9tvTP380PxItGiUv4mI3SXNlPQySZ8oOJ4dOFO3/4d67w9jMk3SxZIOkNQhqU3SZ4atszgidi+Ztpbbke03SVog6Q2S2iW9WNKFJav0SfqZpBdK6pX0TdvT6/hc0IL4IkBDRcTDkr6rLOFK2t6iuLhk/uO219t+yPbc0u1tv9D2v9v+o+2f2L7Y9kBJ+ZG2b7X9e9u/tP23lWKxfYfthbb/U9Izkl5se5rtpXn9D+b7n5SvP8n2pbYfs/0b22fl3dq77OT+DrP9/bxl9Jjta/Lltn2Z7Ufzsrttd1Z4rT5g+/78+d5g+4CSsrD9Qdu/sv2E7S/a9mjeJ9tn2B6yvdH2A7bPrLLuGtufsP3zvJ6v2t5t2DofzZ/PettnlCw/2fbP8vfzd7YvGE18tYiIqyPi5oh4JiKekHSlpON2cndzJC2NiHvzfV0k6XRJsn2EpJdLOj8iNkXEdZLukfR/xvwk0NJItGgo222STpJ0f4XyEyV9TNLxkg6XNLzr7YuSnpa0v7IvvTkl206VdKukqyXtK6lb0r/YfmmVkN4naZ6kPSStlbRC0hZJhylreZ8g6f35uh/IY5+p7Av17WPc30WSbpG0t7JW1uX58hMkvUbSEZL2kvRuSY8Pr8j26yX9k6S/lTQjr+8bw1Z7i6RXSDomX+9NVV6LUo/m2+4p6QxJl9l+eZX135vv+9A87n8oKdtfWavyQEk9kr5oe++87GlJp+XP82RJH7Jd7nWV7YOddf9Wmt4zyuf2Gkn3Dlv24fzHyirb1RLjSyXdVTJ/l6T9bL8wL3sgIjYOK6/2+cNEEBFMTEknSWskPSVpo6SQdJukvUrKl0u6OH+8TNKikrIj8m0OkzRJ0mZJLykpv1jSQP743ZL+Y1jdX1HWwigX1x2SPl0yv5+kZyVNKVnWLak/f3y7pDNLyt6Yx7bLTu7vKklXSGobFtfrJd0n6VWSnjesrPS1Wqqsy3Nb2e7569Oez4ekWSXl10pasJPv4b9JOid//DpJ64a9vx8smX+zpF+XrLtp22uUL3tU0qsq1PN5SZcl/CweL+kJSUeULHu5sq7eXfLYN0o6rsL2v5Z0Ysn85Px1blf2I+tHw9ZfKGl56v8xpuaeaNGiUd4eEXso++I9UtI+FdY7QNLvSubXljyeruzLsLS89PEhkl5Z2spR1tLav0pcw7efLGl9yfZfUdY6Lhdb6eOd2d+5kizpv23fu62bPCJul/QFZa33R2xfYXvPMnUdoJLXJyKeUtbyPbBknYdLHj+jLBmPyPZJtn+Ut/L+oCwBVXrPpOe+ZweUzD8eEVvKxWH7lc4GD22w/aSkD45Qz06z/SplvR3vjIj7ti2PiJ9GxOMRsSUibpL0dUnvqLCbp5S18rfZ9nhjmbJt5RuFCY1Ei4aKiO8ra5V9tsIq6yUdVDJ/cMnjDcq6YdtKlpWu+ztJ34+IvUqm3SPiQ9VCGrb9s5L2Kdl+z4jY1vW3vkrdNe8vIh6OiA9ExAGSzlTWzX1YXvbPEfHXyrodj5D08TJ1PaQsmUva3nX+QkkPVnm+I3I2ivY6Ze/RfhGxl6SblP0oqGT4e/bQKKu7WtINkg6KiGmSvlypnrzr+Kkq03urPKeX5fXMjYjbRogpKsWgrMv5mJL5YyQ9EhGP52UvdsmI5rx8eDc1JhgSLYrweUnH255ZpuxaSafbPsr28yWdv60gspGg35J0ge3n2z5S2fG9bW6UdITt99menE+vsN0xmqAiYr2yY6aX2t7T9vNsH2r7tSWxnWP7QNt7STpvLPuz/a78mLWUdWeGpK15zK+0PVnZMcw/SSo3CvZqSWfYnpknx/8n6ccRsWak5+rsNJSw3V6m+K8k7ar8h43tk5QdN67mI7bbbL9A0iclXTNSDLk9JP0+Iv5k+1hJFY+zRsRvY8eRwcOnr5fbLh9IdrOk+RHx72XK32l79/z9OUHS/1WWlMu5SlJP/vncW9mx6OV5fPdJulPS+bZ3s32KpP+l7EcLJjASLRouIjYo+8J6zoUXIuI7yhLx7coGTN0+bJWzlA2seVjS15SdTvFsvu1GZQnhVGUtqoclXaIsaYzWacoSzc+VJb9vKhtoJGWjVW+RdLeyUzhuUtbCLnsqyCj29wpJP7b9lLIv9nMi4jfKuhuvzNdfq6w7+Dk9AHnL7FPKvsjXKxuIdOoon+dB+b6f0/rNX8ezlf2weEJZ8quUeLa5Wtlr80A+XVx99e0+LOnTtjdK+se8znr7qLLDDktLWr+lrcxzlL0Of1B22s8HIuIOaYdW9MGSFBE3S1osqV/Z67dWJT8Glb3+Xcpet0XKuqk3JHhOaCGO4MbvaF22L5G0f0TMGXHl+td9kqQvR8QhI67cZGz/g6QNEfGVOuxrjaT3R8T3xhwYMA7tUnQAQC3y7uK/UnZ+4iuUnS7y/qob1a/uKZJmK2u57aesJXN9I+qut4gYbYsTwBjRdYxWs4ey47RPK+tmvFTStxtUt5VdBegJZV3HQ8q6OwGgIrqOAQBIiBYtAAAJkWgBAEioqQZD7bPPPtHe3l50GAAA1GTVqlWPRUTZOzU1VaJtb2/X4OBg0WEAAFAT22srldF1DABAQiRaAAASItECAJAQiRYAgIRItAAAJESiBQAgIRItAAAJkWgBAEiIRAsAQEIkWgAAEmqqSzBifLBd8zbcrhHAeEWLFnUXEWWnQ867sWIZAIxXJFoAABIi0QIAkBCJFgCAhEi0AAAkRKIFACAhEi0AAAmRaAEASIhECwBAQlwZCgCwHVd2qz9atACA7biyW/2RaAEASGjMidb2Qbb7bQ/Zvtf2OfnyC2w/aPvOfHrz2MMFAKC11OMY7RZJH42In9reQ9Iq27fmZZdFxGfrUAcAAC1pzIk2ItZLWp8/3mh7SNKBY90vAADjQV2P0dpul/QyST/OF51l+27by2zvXc+6AABoBXVLtLZ3l3SdpL+LiD9K+pKkQyXNVNbivbTCdvNsD9oe3LBhQ73CAQCgKdQl0dqerCzJfj0iviVJEfFIRGyNiL9IulLSseW2jYgrIqIrIrqmT59ej3AAAGgaYz5G6+zs5qWShiLicyXLZ+THbyXpFEmrx1oXmssxF96iJzdtrmmb9gUrR73utCmTddf5J9QaFgA0lXqMOj5O0vsk3WP7znzZJyV1254pKSStkXRmHepCE3ly02atWXRysv3XkpQBoFnVY9TxgKRy1+y6aaz7BgCg1XGtYwCYgDj00zgkWgCYgDj00zhc6xgAgIRo0WKn7dGxQEevWJBw/5KU7hc3ADQCiRY7bePQIrqeAGAEdB0DAJAQiRYAgIRItAAAJMQxWgCYgBjM2DgkWgCYgBjM2Dh0HQMAkBAtWoxJyl+t06ZMTrZvAGgUEi12Wq3dTu0LVibtqgKAZkTXMQAACdGiBYAJikM/jUGiBYAJiEM/jUPXMQAACdGiRd3Zrlx2SfnlEZEoGgAoFokWdUfSBID/QdcxAAAJkWgBAEiIRAsAQEJjTrS2D7Ldb3vI9r22z8mXv8D2rbZ/lf/de+zhAgDQWurRot0i6aMR0SHpVZI+YvsoSQsk3RYRh0u6LZ8HADQx22WntZe8pWIZqhtzoo2I9RHx0/zxRklDkg6U9DZJK/LVVkh6+1jrAgCkFRE1T6iursdobbdLepmkH0vaLyLWS1kylrRvhW3m2R60Pbhhw4Z6hgMAQOHqlmht7y7pOkl/FxF/HO12EXFFRHRFRNf06dPrFQ4AAE2hLonW9mRlSfbrEfGtfPEjtmfk5TMkPVqPugAAaCX1GHVsSUslDUXE50qKbpA0J388R9K3x1oXAACtph6XYDxO0vsk3WP7znzZJyUtknSt7R5Jv5X0rjrUBQBASxlzoo2IAUmVxne/Yaz7BwCglXFlKAAAEiLRAgCQEIkWAICESLQAgIr6+vrU2dmpSZMmqbOzU319fUWH1HK48TsAoKy+vj719vZq6dKlmjVrlgYGBtTT0yNJ6u7uLji61uFmuk5lV1dXDA4OFh0GAEBSZ2enLr/8cs2ePXv7sv7+fs2fP1+rV68uMLLmY3tVRHSVK6PrGBgBXWeYqIaGhjRr1qwdls2aNUtDQ0MFRdSaSLRAFX19fTrnnHP09NNPKyL09NNP65xzziHZYkLo6OjQwMDADssGBgbU0dFRUEStiUQLVHHuuedq0qRJWrZsmZ599lktW7ZMkyZN0rnnnlt0aEByvb296unpUX9/vzZv3qz+/n719PSot7e36NBaCoOhgCrWrVunW265ZfsxqtmzZ+uqq67SCSecUHBkQHrbBjzNnz9fQ0ND6ujo0MKFCxkIVSMSLQCgou7ubhLrGNF1DFTR1tamOXPm7NB1NmfOHLW1tRUdGoAWQaIFqli8eLG2bNmiuXPnarfddtPcuXO1ZcsWLV68uOjQALQIEi1QRXd3t5YsWaKpU6dKkqZOnaolS5bQlQZg1LhgBQAAY1TtghUMhgKGsSvdXrmyZvrBCqC50HUMDBMRZadDzruxYhkAVEKiBQAgIRItAAAJkWgBAEiIRAsAQEJ1SbS2l9l+1PbqkmUX2H7Q9p359OZ61AUAQCupV4t2uaQTyyy/LCJm5tNNdaoLAICWUZdEGxE/kPT7euwLAIDxJPUFK86yfZqkQUkfjYgnEtcHjNoxF96iJzdtrmmb9gUrR73utCmTddf53E4PmOhSJtovSbpIUuR/L5U0d/hKtudJmidJBx98cMJwgB09uWmz1iw6Odn+a0nKAMavZKOOI+KRiNgaEX+RdKWkYyusd0VEdEVE1/Tp01OFAwBAIZK1aG3PiIj1+ewpklZXWx9otD06FujoFQsS7l+S0rWYAbSGuiRa232SXidpH9vrJJ0v6XW2ZyrrOl4j6cx61AXUy8ahRXQdA0iuLok2IsrdnHNpPfYNAEAr48pQAAAkxP1oMaGl7N6dNmVysn0DaB0kWkxYtR6fbV+wMukxXQDjE13HAAAkRKIFACAhEi0AAAmRaAEASIhECwBAQiRaAAASItECAJAQiRYYQV9fnzo7O7V28VvV2dmpvr6+okMC0EIcEUXHsF1XV1cMDg4WHQYmONs1b9NM/0cAGs/2qojoKldGixYYJiK2T21tbZoxY4Zuv/12/fnPf9btt9+uGTNmqK2tbYf1AKASEi1Qxbp167RixQrNnj1bkydP1uzZs7VixQqtW7eu6NAAtAgSLQAACZFogSra2tp02mmnqb+/X5s3b1Z/f79OO+00tbW1FR0agBZBogWqWLx4sbZu3aq5c+dq11131dy5c7V161YtXry46NAAtAgSLVBFd3e3lixZoqlTp8q2pk6dqiVLlqi7u7vo0AC0CE7vAQBgjDi9BwCAgpBoAQBIiEQLAEBCdUm0tpfZftT26pJlL7B9q+1f5X/3rkddAAC0knq1aJdLOnHYsgWSbouIwyXdls8DADCh1CXRRsQPJP1+2OK3SVqRP14h6e31qAsAgFaS8hjtfhGxXpLyv/smrAsAgKZU+GAo2/NsD9oe3LBhQ9HhAABQVykT7SO2Z0hS/vfRcitFxBUR0RURXdOnT08YDgAAjZcy0d4gaU7+eI6kbyesCwCAplSv03v6JP1Q0ktsr7PdI2mRpONt/0rS8fk8AAATyi712ElEVLrC+hvqsX8AAFpV4YOhAAAYz0i0AAAkRKIFACChuhyjBQBkbNe8TTPdFxz1R4sWAOooIspOh5x3Y8UyjG8kWgAAEiLRAgCQEIkWAICESLQAACREogUAICESLQAACZFoAQBIiEQLAEBCJFoAABIi0QIAkBCJFgCAhEi0AAAkRKIFACAhEi0AAAlxP1oA2AnHXHiLnty0uaZt2hesHPW606ZM1l3nn1BrWGhCJFoA2AlPbtqsNYtOTrb/WpIymhuJFgAwbtiueZuISBDJ/0h+jNb2Gtv32L7T9mDq+gAAE1dElJ0OOe/GimWpNapFOzsiHmtQXQAANA1GHQMAkFAjEm1IusX2KtvzGlAfAABNoxFdx8dFxEO295V0q+1fRMQPthXmyXeeJB188MENCAcAgMZxIw4Eb6/MvkDSUxHx2XLlXV1dMTjIeCkAze/oFUcnr+OeOfckr6NV7cx5zLWo9Txm26sioqtcWdIWre2pkp4XERvzxydI+nTKOgGgETYOLeI82gK10nnMqbuO95N0fX5e0y6Sro6ImxPXCQBA00iaaCPiAUnHpKwDAIBmxpWhAGAnpezenTZlcrJ9o7FItACwE2o9Pti+YGXSY4oTzR4dC3T0igUJ9y9J9Xm/SLQAgJbTSoPRuDIUAAAJkWgBAEiIRAsAQEIkWgAAEmIwFACgJbXK6VUkWgBAy2ml06voOgYAICFatACAcSO/tn75skvKL099FzsSLQBg3GjkrV9Hi65jAAASItECAJDQuOw6rtZHX0kzdjcAAFrfuGzRRkTZ6ZDzbqxYBgBACuOyRQsARWnGUa8oFokWAOqIpInhxmXXMQAAzYJECwBAQi3ddXzMhbfoyU2ba9qmlotQT5syWXedf0KtYQEAsF3yRGv7RElLJE2S9K8Rsahe+35y0+akF4lOeWcIAMDEkLTr2PYkSV+UdJKkoyR12z4qZZ0A0Ez6+vrU2dmpSZMmqbOzU319fUWHhAZL3aI9VtL9EfGAJNn+hqS3Sfp54noBoHB9fX3q7e3V0qVLNWvWLA0MDKinp0eS1N3dXXB0aJTUifZASb8rmV8n6ZWVVn788ce1fPnyUe/8xL96XMuXb9jp4IreP4Dx7Sc/+YnOPvtsrV27VmvXrpUknX322RoYGNCzzz5bcHRolNSjjsudub3DSWa259ketD24eXNtA5sAoJk988wzmjZt2g7Lpk2bpmeeeaagiFAEpzy52varJV0QEW/K5z8hSRHxT+XW7+rqisHBwVHv/+gVR9cjzKrumXNP8joAjE+dnZ26/PLLNXv27O3L+vv7NX/+fK1evbrAyFBvtldFRFe5stRdxz+RdLjtF0l6UNKpkt5Tr51vHFrEqGMATau3t1c9PT3POUa7cOHCokNDAyVNtBGxxfZZkr6r7PSeZRFxb8o6AaBZbBvwNH/+fA0NDamjo0MLFy5kINQEk/w82oi4SdJNqesBgGbU3d1NYp3guAQjAAAJtfQlGKW0x1GnTZmcbN8AgImhpRNtrQOh2hesTDp4CgCA4eg6BgAgIRItAAAJkWgBAEiIRAsAQEIkWgAAEiLRAgCQEIkWAICESLQAACTU0hesqMQudxvcvOyS8stT3i4QADBxjctES9IEADQLuo4BAEiIRAsAQEIkWgAAEiLRAgCQEIkWAICESLQAACQ0Lk/vASayaueRV8IpcUA6tGiBcSYiyk6HnHdjxTIA6ZBoAQBIKFmitX2B7Qdt35lPb05VFwAAzSr1MdrLIuKziesAAKBp0XUMAEBCqRPtWbbvtr3M9t6J6wIAoOmMKdHa/p7t1WWmt0n6kqRDJc2UtF7SpRX2Mc/2oO3BDRs2jCUcAOOA7ZonoJmN6RhtRLxxNOvZvlLSjRX2cYWkKySpq6uL8wyACa7S6UbtC1ZqzaKTGxwNMHYpRx3PKJk9RdLqVHUBANCsUo46Xmx7pqSQtEbSmQnrAgCgKSVLtBHxvlT7BgCgVXB6DwAACZFoAQBIiEQLAEBCJFoAABLifrRAizrmwlv05KbNNW3TvmDlqNedNmWy7jr/hFrDAjAMiRZoUU9u2pz0Ag61JGUAldF1DABAQiRaAAASousYQCE4xoyJgkTbhHbmbiSVLsQONCuOMWOioOu4CUVE2emQ826sWAYAaE60aIEWtUfHAh29YkHC/UsSt6UDxopEC7SojUOL6HoFWgBdxwAAJESLFkAh6PrGREGiBVAIur4xUdB1DABAQrRogRaWstU2bcrkZPsGJhISLdCiau12bV+wMmlXLYDy6DoGACAhWrQACkPXNyYCEi2AQtD1jYliTInW9rskXSCpQ9KxETFYUvYJST2Stko6OyK+O5a6xiPuXgIA499YW7SrJb1D0ldKF9o+StKpkl4q6QBJ37N9RERsHWN94wp3LwGA8W9Mg6EiYigiflmm6G2SvhERz0bEbyTdL+nYsdQFAEArSnWM9kBJPyqZX5cvA5BYtfsZ+5Lyy7nVIpDOiInW9vck7V+mqDcivl1pszLLyv4n254naZ4kHXzwwSOFA2AEJE2guYyYaCPijTux33WSDiqZb5P0UIX9XyHpCknq6uriGwIAMK6kumDFDZJOtb2r7RdJOlzSfyeqCwCApjWmRGv7FNvrJL1a0krb35WkiLhX0rWSfi7pZkkfYcQxAGAiGtNgqIi4XtL1FcoWSlo4lv2Pd9yPEwDGP64MVSDuxwkA4x83FQAAICFatAXjourAjjgPGOMNibZAXFQdeC6SJsYbuo4BAEiIRAsAQEIkWgBNra+vT52dnZo0aZI6OzvV19dXdEhATThGC6Bp9fX1qbe3V0uXLtWsWbM0MDCgnp4eSVJ3d3fB0QGjQ4sWQNNauHChli5dqtmzZ2vy5MmaPXu2li5dqoULuRYOWgeJFkDTGhoa0qxZs3ZYNmvWLA0NDRUUEVA7Em0Tsl12WnvJWyqWAeNRR0eHBgYGdlg2MDCgjo6OgiICakeibUIRUfMEjEe9vb3q6elRf3+/Nm/erP7+fvX09Ki3t7fo0IBRYzAUgKa1bcDT/PnzNTQ0pI6ODi1cuJCBUGgpbqbWUFdXVwwODhYdBgAANbG9KiK6ypXRdQwAQEIkWgAAEiLRAgCQEIkWAICESLQAACREogUAICESLQAACZFoAQBIiEQLAEBCTXVlKNsbJK1NWMU+kh5LuP/UiL9YxF8s4i8W8Vd3SERML1fQVIk2NduDlS6R1QqIv1jEXyziLxbx7zy6jgEASIhECwBAQhMt0V5RdABjRPzFIv5iEX+xiH8nTahjtAAANNpEa9ECANBQ4yrR2j7Fdtg+smTZ4bZvtP1r26ts99t+TV52uu0Ntu8smY4qMP5e2/favjuP5ZW277D9y5L4vpmv+8+2PzVs2y8WFXsew9Y8xrts/9T2/86Xt+fvy0Ul6+5je7PtL+TzF9j+WFGxDzfCc9mUl/3c9pdtN+X/UYXP01m278/fj32KjrGaCvEvzd+Tu21/0/buRcdZSbn4S8out/1UkfGNpMLrv9z2b0q+j2YWHWclFeK37YW277M9ZPvshgQTEeNmknStpP+QdEE+v5uk+yS9tWSdTkmn549Pl/SFouPOY3m1pB9K2jWf30fSAZLukNRVZv09JT0g6cWSXiTpN5L2Kvg5PFXy+E2Svp8/bpf0a0k/Kyn/kKQ7t73+ki6Q9LGi34dRPpfV+eNdJP1A0juKjreGz9PL8uewRtI+Rce5E/HvWbLO5yQtKDrWWuLPH3dJ+lrpZ6zZpiqv/3JJ7yw6vjHEf4akqyQ9L1++byPi2UXjRP7L9jhJsyXdoOyL+72SfhgRN2xbLyJWS1pdRIwjmCHpsYh4VpIi4jFJsl125Yj4o+1eSV/IF/1jRPyhEYGO0p6SniiZ3yRpyHZXRAxKereyH0YHFBFcjYY/F0lSRGyx/V+SDmt8SCMq+3mS9JBU+XPVRCrFL0ly9gSmSGrWQSaV/p8nSfqMpPdIOqW48EZU0/dRE6oU/4ckvSci/pIvf7QRwTRll9dOerukmyPiPkm/t/1ySS+V9NMRtnv3sK7jKckjLe8WSQflXRr/Yvu1JWVfL4nvM9sWRkSfpL2V/cr/WqMDLmNKHuMvJP2rpIuGlX9D0qm22yRtVf6l36RGei6y/XxJb5B0T6ODG4Vqn6dWUDF+21+V9LCkIyVdXlSAI6gU/1mSboiI9QXGNhrVPj8L8+7Yy2zvWlSAI6gU/6HKvvMHbX/H9uGNCGY8JdpuZV/kyv92D1/B9vW2V9v+VsniayJiZsm0qRHBDhcRT0n6a0nzJG2QdI3t0/Pi95bE9/Ft2+QJa39JBzTJsapNeYxHSjpR0lXe8SfwzZKOV/beXFNEgDWo9lwOtX2npP+UtDIivlNYlBWM8HlqetXij4gzlPWEDCnrGWk6FeL/pKR3qXl/HGxX5fX/hLIfOK+Q9AJJ5xUVYzVV4t9V0p8iu0LUlZKWNSKecdF1bPuFkl4vqdN2SJqkrEvpQkmv2bZeRJxiu0vSZwsJdAQRsVXZMdk7bN8jac4ImyxR1kXeIel8SR+vunYDRcQP88E200uW/dn2KkkfVdbb8DdFxVeLMs/l1xHRtINAtqnweVpeZEy1qBZ/RGy1fY2yz/xXi4qxmjLx90l6XNL9+W+259u+PyKa8dBD2dc/Ipbnxc/mPQtNM4BxuAqfn3WSrstXuV4N+uyMlxbtOyVdFRGHRER7RBykbHDQfZKOs/3WknWfX0iEI7D9kmHdGDNV5QYLtk+StK+yA/sXSTrFBY6YHs7ZyO9Jyr5YSl0q6byIGL68aVV5Lk2r1s9Ts6kQ/29tH5aXW9kPtV8UEd9IKsT/lYjYP/+Oapf0TLMm2UqfH9sz8nIrO1zXjONdqn3+/01Zo0ySXqssRyQ3Llq0yroiFw1bdp2yAQdvkfQ525+X9IikjZIuLlnv3f0tFPEAAADVSURBVLZnlcx/OCL+K2WwFewu6XLbe0naIul+Zd0e31R2jHZbl/Zjyp7T55WN/gtJT9s+V9nAqNc/Z8+NMyXvUpUkK/sFvLW09zgi7pV0bxHB1WjE59Lkyn6e8tMZzlV2yOFu2zdFxPsLjLOScvF/UNL1tvdU9p7cpWz0ejOq9P/cKirFf63t6cpe/zuVvSfNqFL8W5R9n/69pKckNeSzz5WhAABIaLx0HQMA0JRItAAAJESiBQAgIRItAAAJkWgBAEiIRAsAQEIkWgAAEiLRAgCQ0P8HhFxuROfOvI4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Xn = nX.drop(columns=['S2'])\n",
    "ridge_coefs_analyse(Xn, Y, 25., 5, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop the `S4` column. Seems to give the best accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing score is 0.4903 ± 0.0531\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAE/CAYAAADhbQKeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfUElEQVR4nO3de7RcdX338ffHGDWWiyhRuaci1Gh8QHu8PfJY8YK3toqPVlMfizaKWqW2y1qjaQteaEPrrUVbjYUGuzTqUqkUKOIlamOtbVBAMPWOFQ0SUDFotIF+nz/2jh2OZ84lZ/aZmZP3a61ZZ2b/9uz9ncuZz+zf/u3ZqSokSVI3bjfsAiRJWswMWkmSOmTQSpLUIYNWkqQOGbSSJHXIoJUkqUMGrYYuyduS/PE07ZXk3gtZ06ia6blaoBoemeTaQc8rLVYGrTqX5Joku5LcnOS6JBuT7LenvapeWFWvHWaN48Lnau6SPCnJliQ/aN9/70iyf0/7xiT/1b4/91yWTLO832+Xc1OSc5PcsadtRZLNSX6c5D+SPKbrx6fRZ9BqofxaVe0HHA88AHjlkOu5jTQG9v8w6OVpXg4EXgccCqwEDgf+YtI8f15V+/Vcbp1qQUkeB6wFHg2sAO4FvLpnlk3A54G7AeuA9ydZPsDHojHkB4EWVFVdB3yYJnCBn21RvK7n9suTbE/ynSS/3Xv/JHdL8o9Jfpjk35O8LsmWnvb7JPlIku8l+VKS3+hXS5JPJDkzyaeBHwP3SnJgknPa9X+7Xf6Sdv4lSd6Q5IYk30jykrZb+/Z7ubx7J/lku2V0Q5L3ttOT5E1Jrm/brkyyqs9z9fwkX20f7wVJDu1pqyQvTPKVJN9P8tYkmc3rlOS5SbYl2Znk60leMM281yR5ZZIvtuv5uyR3mjTPy9rHsz3Jc3umPynJ59vX81tJzphNfXNRVe+uqkuq6sdV9X3gHcDD93JxpwDnVNXV7bJeCzwHIMmxwAOB06tqV1V9APgC8H/n/SA01gxaLagkhwNPAL7ap/3xwB8AjwWOASZ3vb0V+BFwT5oPvVN67vsLwEeAdwN3B1YDf53kftOU9GzgVGB/4JvAecAtwL1ptrxPAp7Xzvv8tvbjaT5QnzLP5b0WuBQ4iGYr6+x2+knAI4BjgbsAzwBunLyiJI8C/gz4DeCQdn3vmTTbrwIPAo5r53vcNM9Fr+vb+x4APBd4U5IHTjP/s9plH93W/Uc9bfek2ao8DFgDvDXJQW3bj4Dfah/nk4AXJZnqeSXJkWm6f/tdfnOWj+0RwNWTpv1O+2XlsiTTBeP9gCt6bl8B3CPJ3dq2r1fVzknt073/tC+oKi9eOr0A1wA3AzuBAj4G3KWnfSPwuvb6ucD6nrZj2/vcG1gC7AZ+qaf9dcCW9vozgH+etO6302xhTFXXJ4DX9Ny+B/BTYFnPtNXA5vb6x4EX9LQ9pq3t9nu5vHcCG4DDJ9X1KODLwEOB201q632uzqHp8tzTtl/7/KxobxdwQk/7+4C1e/ka/gPw0vb6I4FrJ72+L+y5/UTgaz3z7trzHLXTrgce2mc9bwbe1OF78bHA94Fje6Y9kKar9/Zt7TuBh/e5/9eAx/fcXto+zytovmT966T5zwQ2dv0/5mW0L27RaqE8par2p/ngvQ9wcJ/5DgW+1XP7mz3Xl9N8GPa2914/CnhI71YOzZbWPaepa/L9lwLbe+7/dpqt46lq672+N8v7QyDAvyW5ek83eVV9HHgLzdb7d5NsSHLAFOs6lJ7np6puptnyPaxnnut6rv+YJoxnlOQJSf613cr7AU0A9XvN4Odfs0N7bt9YVbdMVUeSh6QZPLQjyU3AC2dYz15L8lCa3o6nVdWX90yvqs9V1Y1VdUtVXQy8C3hqn8XcTLOVv8ee6zunaNvTvhPt0wxaLaiq+iTNVtnr+8yyHTii5/aRPdd30HTDHt4zrXfebwGfrKq79Fz2q6oXTVfSpPv/FDi45/4HVNWerr/t06x7zsurquuq6vlVdSjwAppu7nu3bX9VVb9M0+14LPDyKdb1HZowB37WdX434NvTPN4ZpRlF+wGa1+geVXUX4GKaLwX9TH7NvjPL1b0buAA4oqoOBN7Wbz1t1/HN01yeNc1jekC7nt+uqo/NUFP1q4Gmy/m4ntvHAd+tqhvbtnulZ0Rz2z65m1r7GINWw/Bm4LFJjp+i7X3Ac5LcN8mdgdP3NFQzEvSDwBlJ7pzkPjT79/a4EDg2ybOTLG0vD0qycjZFVdV2mn2mb0hyQJLbJTk6ya/01PbSJIcluQvwivksL8nT233W0HRnFnBrW/NDkiyl2Yf5E2CqUbDvBp6b5Pg2HP8U+GxVXTPTY01zGEolWTFF8x2AO9J+sUnyBJr9xtN5cZLDk9wVeBXw3plqaO0PfK+qfpLkwUDf/axV9Z9125HBky/vmup+7UCyS4DTquofp2h/WpL92tfnJOD/0YTyVN4JrGnfnwfR7Ive2Nb3ZeBy4PQkd0pyMvC/aL60aB9m0GrBVdUOmg+sn/vhhar6J5og/jjNgKmPT5rlJTQDa64D/p7mcIqftvfdSRMIz6TZoroOOIsmNGbrt2iC5os04fd+moFG0IxWvRS4kuYQjotptrCnPBRkFst7EPDZJDfTfLC/tKq+QdPd+I52/m/SdAf/XA9Au2X2xzQf5NtpBiI9c5aP84h22T+39ds+j79L88Xi+zTh1y949ng3zXPz9fbyuuln/5nfAV6TZCfwJ+06B+1lNLsdzunZ+u3dynwpzfPwA5rDfp5fVZ+A22xFHwlQVZcAfw5spnn+vknPl0Ga53+C5nlbT9NNvaODx6QxkipP/K7xleQs4J5VdcqMMw9+3U8A3lZVR80484hJ8kfAjqp6+wCWdQ3wvKr66LwLkxah2w+7AGku2u7iO9Acn/ggmsNFnjftnQa37mXAiTRbbveg2ZI5fyHWPWhVNdstTknzZNexxs3+NPtpf0TTzfgG4EMLtO7Q/ArQ92m6jrfRdHdKUl92HUuS1CG3aCVJ6pBBK0lSh0ZqMNTBBx9cK1asGHYZkiTNyWWXXXZDVU15pqaRCtoVK1awdevWYZchSdKcJPlmvza7jiVJ6pBBK0lShwxaSZI6ZNBKktQhg1aSpA4ZtJIkdciglSSpQwatJEkdMmglSerQvIM2yRFJNifZluTqJC9tp5+R5NtJLm8vT5x/uZIkjZdB/ATjLcDLqupzSfYHLkvykbbtTVX1+gGsQ2MkyZzv4+kaJS1W896irartVfW59vpOmpNhHzbf5Wp8VdWUl6NecWHfNklarAa6jzbJCuABwGfbSS9JcmWSc5McNMh1SZIGL8mcL5rewII2yX7AB4Dfq6ofAn8DHA0cD2wH3tDnfqcm2Zpk644dOwZVjiRpL9gjNXgDOU1ekqU0IfuuqvogQFV9t6f9HcCFU923qjYAGwAmJiZ8xcbIca++lJt27Z7TfVasvWjW8x64bClXnH7SXMuSpJEy76BN029wDrCtqt7YM/2Qqtre3jwZuGq+69JouWnXbq5Z/6TOlj+XUJakUTWILdqHA88GvpDk8nbaq4DVSY4HCrgGeMEA1iVJ0liZd9BW1RZgqr3hF8932ZIkjTt/GUqSpA4ZtJIkdWggo44lSePFowYWjkGrvbb/yrXc/7y1HS4foLtRzdK+zKMGFo5Bq722c9t6/1ElaQbuo5UkqUMGrSRJHbLrWPPSZffugcuWdrZsaV/nGIuFY9Bqr811/+yKtRd1uk9X0uw5xmLh2HUsSVKHDFpJkjpk0EqS1CGDVpKkDhm0kiR1yFHHkrSP8vC8hWHQStI+yMPzFo5dx5IkdciglSSpQwatJEkdMmglSeqQg6E0cEn6t5019fSq6qgaSRoug1YDZ2hK0v+w61iSpA7NO2iTHJFkc5JtSa5O8tJ2+l2TfCTJV9q/B82/XEmSxssgtmhvAV5WVSuBhwIvTnJfYC3wsao6BvhYe1uSpH3KvIO2qrZX1efa6zuBbcBhwJOB89rZzgOeMt91SZI0bga6jzbJCuABwGeBe1TVdmjCGLj7INclSdI4GFjQJtkP+ADwe1X1wznc79QkW5Ns3bFjx6DKkSRpJAzk8J4kS2lC9l1V9cF28neTHFJV25McAlw/1X2ragOwAWBiYsLjQiRpiDwOfvAGMeo4wDnAtqp6Y0/TBcAp7fVTgA/Nd12SpG5V1Zwvmt4gtmgfDjwb+EKSy9tprwLWA+9Lsgb4T+DpA1iXJEljZd5BW1VbgH59DY+e7/IlSRpn/jKUJEkdMmglSeqQQStJUocMWkmSOmTQSpLUIYNWkqQOGbSSJHXIoJUkqUMGrSRJHTJoJUnq0EDO3iMtJtOdvaQff1hdUj9u0UqT9DtDyVGvuNCzl0iaM4NWkqQOGbSSJHXIoJUkqUMGrSRJHTJoJUnqkEErSVKHDFpJkjpk0EqS1CGDVpKkDhm0kiR1yKCVJKlDAzmpQJJzgV8Frq+qVe20M4DnAzva2V5VVRcPYn3SIBz36ku5adfuOd1nxdqLZj3vgcuWcsXpJ821LEmLzKDO3rMReAvwzknT31RVrx/QOqSBumnXbq5Z/6TOlj+XUJa0eA2k67iqPgV8bxDLkiRpMel6H+1LklyZ5NwkB001Q5JTk2xNsnXHjh1TzSJJ0tjqMmj/BjgaOB7YDrxhqpmqakNVTVTVxPLlyzssR5KkhTeofbQ/p6q+u+d6kncAF3a1Lmlv7L9yLfc/b22Hywfobh+wpPHQWdAmOaSqtrc3Twau6mpd0t7YuW29g6EkdW5Qh/dsAh4JHJzkWuB04JFJjgcKuAZ4wSDWJUnSOBlI0FbV6ikmnzOIZUuSNM78ZShJkjpk0EqS1CGDVpKkDnU26lgaB12ODD5w2dLOli1pfBi02mfN9dCeFWsv6vRwIEmLk13HkiR1yKCVJKlDBq0kSR0yaCVJ6pBBK0lShwxaSZI6ZNBKktQhg1aSpA4ZtJIkdchfhpImSdK/7aypp1dVR9VIGncGrTSJoSlpkOw6liSpQwatJEkdMmglSeqQQStJUocMWkmSOmTQSpLUoYEEbZJzk1yf5KqeaXdN8pEkX2n/HjSIdUmSNE4GtUW7EXj8pGlrgY9V1THAx9rbkiTtUwYStFX1KeB7kyY/GTivvX4e8JRBrEuSpHHS5T7ae1TVdoD27907XJckSSNp6IOhkpyaZGuSrTt27Bh2OZIkDVSXQfvdJIcAtH+vn2qmqtpQVRNVNbF8+fIOy5EkaeF1GbQXAKe0108BPtThuiRJGkmDOrxnE/AZ4JeSXJtkDbAeeGySrwCPbW9LkrRPGchp8qpqdZ+mRw9i+ZIkjauhD4aSJGkxM2glSeqQQStJUocMWkmSOmTQSpLUIYNWkqQOGbSSJHXIoJUkqUMGrSRJHTJoJUnqkEErSVKHDFpJkjpk0EqS1CGDVpKkDhm0kiR1yKCVJKlDBq0kSR0yaCVJ6tDth12AJEmDkmTO96mqDir5H27RSpIWjaqa8nLUKy7s29Y1g1aSpA4ZtJIkdciglSSpQ50PhkpyDbATuBW4paomul6nJEmjYqFGHZ9YVTcs0LokSRoZdh1LktShhQjaAi5NclmSUyc3Jjk1ydYkW3fs2LEA5UiStHAWImgfXlUPBJ4AvDjJI3obq2pDVU1U1cTy5csXoBxJkhZO50FbVd9p/14PnA88uOt1SpI0KjodDJXkF4DbVdXO9vpJwGu6XKckafE77tWXctOu3XO6z4q1F8163gOXLeWK00+aa1lT6nrU8T2A89vfnrw98O6quqTjdUqSFrmbdu3mmvVP6mz5cwnlmXQatFX1deC4LtchSdIo8/AeSZI65GnyJEljZ/+Va7n/eWs7XD7AYLqmDVpJ0tjZuW392OyjtetYkqQOuUUrSRpLg9zqnOzAZUsHtiyDVpI0dubabbxi7UWddjVPx65jSZI6ZNBKktQhg1aSpA4ZtJIkdciglSSpQwatJEkdMmglSeqQx9FKkhaN9rSsU7edNfX0quqomoZBK0laNLoOzb1h17EkadHatGkTq1atYsmSJaxatYpNmzYteA1u0UqSFqVNmzaxbt06zjnnHE444QS2bNnCmjVrAFi9evWC1ZFR2syemJiorVu3DrsMSdIisGrVKs4++2xOPPHEn03bvHkzp512GlddddVA15XksqqamLLNoJUkLUZLlizhJz/5CUuX/s+ZeHbv3s2d7nQnbr311oGua7qgdR+tJGlRWrlyJVu2bLnNtC1btrBy5coFrWNRBm2SOV8kjQb/fzUo69atY82aNWzevJndu3ezefNm1qxZw7p16xa0jkU5GKpfd/gwz0coaXb8/9Wg7BnwdNppp7Ft2zZWrlzJmWeeuaADoWABgjbJ44G/BJYAf1tV67tepyRJ0ITtQgfrZJ0GbZIlwFuBxwLXAv+e5IKq+uIgln/cqy/lpl2753SfFWsvmvW8By5byhWnnzTXsiRJ+pmut2gfDHy1qr4OkOQ9wJOBgQTtTbt2d9qVNJdQliRpKl0H7WHAt3puXws8pN/MN954Ixs3bpz1wh9/hxvZuHHHXhc37OVLmj3/HzWuug7aqYYD3makQ5JTgVMBDjvssDkt/NNHb+TTe13aLBwND62XdbkGSdIi1+kPViR5GHBGVT2uvf1KgKr6s6nmn+sPVnQ9CtFRjtLo8P9Ro2yYP1jx78AxSX4xyR2AZwIXdLxOSZJGRqddx1V1S5KXAB+mObzn3Kq6ust1SpI0Sjo/jraqLgYu7no9ksaLh+dpXzH2vwzV5SE4By5bOvNMkvaKh+dpXzHWQTvXf1IHU0iSFtqiPKmAJEmjwqCVJKlDY911LOnn7c1p47o8nr6f/Veu5f7nre1w+QDuKtLwGbTSIjMup5nbuW29g6G0T1iUQTvdN/qcNfX0YXyjlyQtfosyaA1NSdKoWJRBK2k8eBy89gUGraSh8Dh47Ss8vEeSpA4ZtJIkdciuY2lM+aP80ngwaKUx5Y/yS+PBoJU0UjwOXouNQStppBiaWmwcDCVJUocMWkmSOmTXsTSmPPuNNB4MWmlMefYbaTzYdSxJUocMWkmSOmTQSpLUoc6CNskZSb6d5PL28sSu1iVJ0qjqeov2TVV1fHu5uON1SVqENm3axKpVq1iyZAmrVq1i06ZNwy5JmhNHHUtjbLGfOH3Tpk2sW7eOc845hxNOOIEtW7awZs0aAFavXj3k6qTZSVc/d5bkDOA5wA+BrcDLqur7091nYmKitm7d2kk90r5uHE+cvmrVKs4++2xOPPHEn03bvHkzp512GlddddUQK5NuK8llVTUxVdu8uo6TfDTJVVNcngz8DXA0cDywHXhDn2WcmmRrkq07duyYTzmSFplt27Zxwgkn3GbaCSecwLZt24ZUkTR38+o6rqrHzGa+JO8ALuyzjA3ABmi2aOdTj6TFZeXKlWzZsuU2W7Rbtmxh5cqVQ6xKmpsuRx0f0nPzZMB+Hklzsm7dOtasWcPmzZvZvXs3mzdvZs2aNaxbt27YpUmz1uVgqD9PcjxQwDXACzpcl6RFaM+Ap9NOO41t27axcuVKzjzzTAdCaax0FrRV9eyuli1p37F69WqDVWPNX4aSJKlDBq0kSR3yByukRSZJ/7azpp7e1fH0kgxaadExNKXRYtexJEkdMmglSeqQQStJUofcRzuCphvM0o/75SRpNLlFO4KqasrLUa+4sG+bJGk0GbSSJHXIoJUkqUMGrSRJHTJoJUnqkEErSVKHDFpJkjpk0EqS1CGDVpKkDhm0kiR1yKCVJKlD/tbxEB336ku5adfuOd1nxdqLZj3vgcuWcsXpJ821LEnSABm0Q3TTrt1cs/5JnS1/LqEsSeqGQStJA+TZtzTZvPbRJnl6kquT/HeSiUltr0zy1SRfSvK4+ZUpSePBs29psvlu0V4FPBV4e+/EJPcFngncDzgU+GiSY6vq1nmuT5KksTKvLdqq2lZVX5qi6cnAe6rqp1X1DeCrwIPnsy5JksZRV4f3HAZ8q+f2te00SZL2KTN2HSf5KHDPKZrWVdWH+t1timlT7ohIcipwKsCRRx45UzmSJI2VGYO2qh6zF8u9Fjii5/bhwHf6LH8DsAFgYmJinxoVsP/Ktdz/vLUdLh+gu8OHJEkz6+rwnguAdyd5I81gqGOAf+toXWNr57b1HkcrSYvcfA/vOTnJtcDDgIuSfBigqq4G3gd8EbgEeLEjjiVJ+6J5bdFW1fnA+X3azgTOnM/yJUkad/4y1JB12b174LKlnS1b2tf5W+WaLYN2iOa6f3bF2os63acrafb8rXLNlqfJkySpQwatJEkdMmglSeqQQStJUocMWkmSOmTQSpLUIYNWkqQOeRytJO0FTwqi2TJoJWkveFIQzZZdx5IkdciglSSpQ3YdS9Je8qQgmg2DVpL2gicF0WwZtCMoSf+2s6aeXlUdVSNJmg+DdgQZmpK0eDgYSpKkDrlFK0kD5K4fTWbQStIAGZqazK5jSZI6ZNBKktQhg1aSpA7NK2iTPD3J1Un+O8lEz/QVSXYluby9vG3+pUqSNH7mOxjqKuCpwNunaPtaVR0/z+VLkjTW5hW0VbUNph/OLknSvqzLfbS/mOTzST6Z5P90uB5JkkbWjFu0ST4K3HOKpnVV9aE+d9sOHFlVNyb5ZeAfktyvqn44xfJPBU4FOPLII2dfuSRJY2DGoK2qx8x1oVX1U+Cn7fXLknwNOBbYOsW8G4ANABMTEx7pLUlaVDrpOk6yPMmS9vq9gGOAr3exLkmSRtl8D+85Ocm1wMOAi5J8uG16BHBlkiuA9wMvrKrvza9USZLGT0bpdzmT7AC+2eEqDgZu6HD5XbP+4bL+4bL+4bL+6R1VVcunahipoO1akq1VNTHznKPJ+ofL+ofL+ofL+veeP8EoSVKHDFpJkjq0rwXthmEXME/WP1zWP1zWP1zWv5f2qX20kiQttH1ti1aSpAW1qIK2Pa63ktynZ9oxSS5M8rUklyXZnOQRbdtzkuzoOZ3f5UnuO8T617WnHbyyreUhST6R5Es99b2/nfevkvzxpPu+dVi1tzXc2tZ4RZLPJfnf7fQV7evy2p55D06yO8lb2ttnJPmDYdU+2QyPZc8pIL+Y5G1JRvL/qM/76SVJvtq+HgcPu8bp9Kn/Xe3/w1VJzk2ydNh19tOn/nPa99SVSd6fZL9h19lPn/o3JvlGz+fRyJ6hrU/9SXJmki8n2ZbkdxekmKpaNBfgfcA/A2e0t+8EfBn49Z55VgHPaa8/B3jLsOtua3kY8Bngju3tg4FDgU8AE1PMfwDNr23dC/hF4BvAXYb8GG7uuf444JPt9RXA14DP97S/CLh8z/MPnAH8wbBfh1k+lqva67cHPgU8ddj1zuH99ID2MVwDHDzsOvei/icCaS+bgBcNu9Y51n9AzzxvBNYOu9Y51r8ReNqw65tH/c8F3gncrp1+94WoZ77nox0Z7TfDhwMnAhfQfHA/C/hMVV2wZ76quormPLqj5hDghmp+J5qqugH6n4Kwqn6YZB3wlnbSn1TVDxai0Fk6APh+z+1dwLYkE1W1FXgGzRejQ4dR3BxNfiwAVNUtSf4FuPfClzSjKd9PwHdgLE5tOW39AEn+DTh8CLXNRr/6AUjzAiwDRnWQzJw+j0ZQv/pfBPxmVf13O/36hShmJLu89tJTgEuq6svA95I8ELgf8LkZ7veMSV3HyzqvdGqXAke0XRp/neRXetre1VPfX+yZWFWbgINoviX//UIXPIVlbY3/Afwt8NpJ7e8BnpnkcOBWej40R9BMj4UkdwYeDXxhoYubheneT+Ng2vrbLuNnA5cMpbqZ9a0/yd8B1wH3Ac4eVoEzmO75P7Ptjn1TkjsOq8AZ9Kv/aJrP/K1J/inJMQtRzGIK2tU0H+S0f1dPniHJ+e2+nQ/2TH5vVR3fc9m1EMVOVlU3A79Mc8rAHcB7kzynbX5WT30v33OfNrDuCRw6Ivt6drU13gd4PPDO3PYr8CXAY2lem/cOo8A5mO6xHJ3kcuDTwEVV9U9Dq7KPGd5PI28W9f818Kmq+uchlDej6eqvqufS9ORso+nZGTnT1P9Kmi8IDwLuCrxiWDVOZ5r67wj8pJpfiHoHcO5C1LMouo6T3A14FLAqSQFLaLpkXk1zggMAqurkJBPA64dS6Ayq6laafbKfSPIF4JQZ7vKXNF3kK4HTgZdPO/cCqqrPtINtlvdM+68klwEvo+lt+LVh1TcXUzyWr1XVyA4C2aPP+2njMGuai371Jzmd5rV4wRDLm9F0z39V3ZrkvTT/s383rBqnM1X9VbWxbf5pu2U+MgMYJ+vz/F8LfKCd5XwW6LlfLFu0TwPeWVVHVdWKqjqCZnDQl4GHJ/n1nnnvPJQKZ5DklyZ1YxzPNCdYSPIE4O40O/ZfC5ycIY6YnizNyO8lwI2Tmt4AvKKqJk8fWdM8lpE11/fTqOlXf5Ln0QxOW71nP9so6lP/fya5d9semi+a/zGM+mYyzfN/SNsemt11ozjeZbr3/z/QbJQB/ApNRnRuUWzR0nRFrp807QPAbwK/CrwxyZuB7wI7gdf1zPeMJCf03P6dqvqXLovtYz/g7CR3AW4BvkrT7fF+mn20e7q0b6B5TG+mGf1XwI+S/CHNwKhH/dySF86ytksVmlGhp7Tf3H82Q1VdDVw9jOLmaMbHMuKmfD+1hzP8Ic0uhyuTXFxVzxtinf30+3+4juYD8zPta/HBqnrN0Krsb6r6Xwicn+QAmvfUFTSj70dRv+f/fUmW09R/Oc1jGkX96r+F5vP094GbgQV57/vLUJIkdWixdB1LkjSSDFpJkjpk0EqS1CGDVpKkDhm0kiR1yKCVJKlDBq0kSR0yaCVJ6tD/B/rnaEHMgGTOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Xn = nX.drop(columns=['S4'])\n",
    "ridge_coefs_analyse(Xn, Y, 25., 5, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
Simon Clarke's avatar
Simon Clarke committed
1300
   "version": "3.7.4"
Simon Clarke's avatar
Simon Clarke committed
1301
1302
1303
1304
1305
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}