loading config file: /content/mycar/config.py
loading personal config over-rides
config loaded
"get_model_by_type" model Type is: categorical
training with model type <class 'donkeycar.parts.keras.KerasCategorical'>
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
img_in (InputLayer) [(None, 120, 160, 3) 0
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 58, 78, 24) 1824 img_in[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 58, 78, 24) 0 conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 27, 37, 32) 19232 dropout_6[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout) (None, 27, 37, 32) 0 conv2d_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 12, 17, 64) 51264 dropout_7[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 5, 8, 64) 36928 conv2d_3[0][0]
__________________________________________________________________________________________________
dropout_8 (Dropout) (None, 5, 8, 64) 0 conv2d_4[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 3, 6, 64) 36928 dropout_8[0][0]
__________________________________________________________________________________________________
dropout_9 (Dropout) (None, 3, 6, 64) 0 conv2d_5[0][0]
__________________________________________________________________________________________________
flattened (Flatten) (None, 1152) 0 dropout_9[0][0]
__________________________________________________________________________________________________
fc_1 (Dense) (None, 100) 115300 flattened[0][0]
__________________________________________________________________________________________________
dropout_10 (Dropout) (None, 100) 0 fc_1[0][0]
__________________________________________________________________________________________________
fc_2 (Dense) (None, 50) 5050 dropout_10[0][0]
__________________________________________________________________________________________________
dropout_11 (Dropout) (None, 50) 0 fc_2[0][0]
__________________________________________________________________________________________________
angle_out (Dense) (None, 15) 765 dropout_11[0][0]
__________________________________________________________________________________________________
throttle_out (Dense) (None, 20) 1020 dropout_11[0][0]
==================================================================================================
Total params: 268,311
Trainable params: 268,311
Non-trainable params: 0
__________________________________________________________________________________________________
None
found 0 pickles writing json records and images in tub /content/mycar/data/tub_6_19-07-29
found 0 pickles writing json records and images in tub /content/mycar/data/tub_7_19-07-29
/content/mycar/data/tub_6_19-07-29
/content/mycar/data/tub_7_19-07-29
collating 10269 records ...
train: 8215, val: 2054
total records: 10269
steps_per_epoch 64
Epoch 1/100
63/64 [============================>.] - ETA: 0s - loss: 3.3674 - angle_out_loss: 2.2257 - throttle_out_loss: 2.2545 - angle_out_acc: 0.2696 - throttle_out_acc: 0.3826
Epoch 00001: val_loss improved from inf to 3.13174, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 12s 184ms/step - loss: 3.3630 - angle_out_loss: 2.2232 - throttle_out_loss: 2.2514 - angle_out_acc: 0.2706 - throttle_out_acc: 0.3831 - val_loss: 3.1317 - val_angle_out_loss: 1.9923 - val_throttle_out_loss: 2.1356 - val_angle_out_acc: 0.3462 - val_throttle_out_acc: 0.3809
Epoch 2/100
63/64 [============================>.] - ETA: 0s - loss: 3.1067 - angle_out_loss: 2.0148 - throttle_out_loss: 2.0993 - angle_out_acc: 0.3323 - throttle_out_acc: 0.3932
Epoch 00002: val_loss improved from 3.13174 to 2.98889, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 53ms/step - loss: 3.1063 - angle_out_loss: 2.0139 - throttle_out_loss: 2.0994 - angle_out_acc: 0.3337 - throttle_out_acc: 0.3938 - val_loss: 2.9889 - val_angle_out_loss: 1.9188 - val_throttle_out_loss: 2.0295 - val_angle_out_acc: 0.3452 - val_throttle_out_acc: 0.3862
Epoch 3/100
63/64 [============================>.] - ETA: 0s - loss: 3.0231 - angle_out_loss: 1.9465 - throttle_out_loss: 2.0499 - angle_out_acc: 0.3960 - throttle_out_acc: 0.4428
Epoch 00003: val_loss improved from 2.98889 to 2.92830, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 3.0233 - angle_out_loss: 1.9471 - throttle_out_loss: 2.0497 - angle_out_acc: 0.3971 - throttle_out_acc: 0.4431 - val_loss: 2.9283 - val_angle_out_loss: 1.8372 - val_throttle_out_loss: 2.0097 - val_angle_out_acc: 0.4624 - val_throttle_out_acc: 0.4546
Epoch 4/100
62/64 [============================>.] - ETA: 0s - loss: 2.8735 - angle_out_loss: 1.8158 - throttle_out_loss: 1.9656 - angle_out_acc: 0.4970 - throttle_out_acc: 0.4946
Epoch 00004: val_loss improved from 2.92830 to 2.69279, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.8727 - angle_out_loss: 1.8150 - throttle_out_loss: 1.9652 - angle_out_acc: 0.4966 - throttle_out_acc: 0.4950 - val_loss: 2.6928 - val_angle_out_loss: 1.6640 - val_throttle_out_loss: 1.8608 - val_angle_out_acc: 0.5469 - val_throttle_out_acc: 0.5000
Epoch 5/100
62/64 [============================>.] - ETA: 0s - loss: 2.7645 - angle_out_loss: 1.7299 - throttle_out_loss: 1.8995 - angle_out_acc: 0.5219 - throttle_out_acc: 0.5150
Epoch 00005: val_loss improved from 2.69279 to 2.65042, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 52ms/step - loss: 2.7651 - angle_out_loss: 1.7300 - throttle_out_loss: 1.9001 - angle_out_acc: 0.5222 - throttle_out_acc: 0.5144 - val_loss: 2.6504 - val_angle_out_loss: 1.6285 - val_throttle_out_loss: 1.8362 - val_angle_out_acc: 0.5547 - val_throttle_out_acc: 0.5249
Epoch 6/100
63/64 [============================>.] - ETA: 0s - loss: 2.6785 - angle_out_loss: 1.6660 - throttle_out_loss: 1.8455 - angle_out_acc: 0.5433 - throttle_out_acc: 0.5295
Epoch 00006: val_loss improved from 2.65042 to 2.55639, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.6771 - angle_out_loss: 1.6656 - throttle_out_loss: 1.8443 - angle_out_acc: 0.5436 - throttle_out_acc: 0.5295 - val_loss: 2.5564 - val_angle_out_loss: 1.5467 - val_throttle_out_loss: 1.7830 - val_angle_out_acc: 0.5791 - val_throttle_out_acc: 0.5322
Epoch 7/100
62/64 [============================>.] - ETA: 0s - loss: 2.6023 - angle_out_loss: 1.6173 - throttle_out_loss: 1.7936 - angle_out_acc: 0.5557 - throttle_out_acc: 0.5387
Epoch 00007: val_loss improved from 2.55639 to 2.53856, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.6040 - angle_out_loss: 1.6213 - throttle_out_loss: 1.7933 - angle_out_acc: 0.5535 - throttle_out_acc: 0.5383 - val_loss: 2.5386 - val_angle_out_loss: 1.5468 - val_throttle_out_loss: 1.7651 - val_angle_out_acc: 0.5542 - val_throttle_out_acc: 0.5347
Epoch 8/100
63/64 [============================>.] - ETA: 0s - loss: 2.5492 - angle_out_loss: 1.5788 - throttle_out_loss: 1.7598 - angle_out_acc: 0.5590 - throttle_out_acc: 0.5484
Epoch 00008: val_loss improved from 2.53856 to 2.48855, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.5489 - angle_out_loss: 1.5782 - throttle_out_loss: 1.7598 - angle_out_acc: 0.5591 - throttle_out_acc: 0.5487 - val_loss: 2.4886 - val_angle_out_loss: 1.5071 - val_throttle_out_loss: 1.7350 - val_angle_out_acc: 0.5752 - val_throttle_out_acc: 0.5439
Epoch 9/100
63/64 [============================>.] - ETA: 0s - loss: 2.4850 - angle_out_loss: 1.5364 - throttle_out_loss: 1.7168 - angle_out_acc: 0.5671 - throttle_out_acc: 0.5530
Epoch 00009: val_loss improved from 2.48855 to 2.45861, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.4876 - angle_out_loss: 1.5366 - throttle_out_loss: 1.7193 - angle_out_acc: 0.5669 - throttle_out_acc: 0.5521 - val_loss: 2.4586 - val_angle_out_loss: 1.4642 - val_throttle_out_loss: 1.7265 - val_angle_out_acc: 0.5908 - val_throttle_out_acc: 0.5547
Epoch 10/100
63/64 [============================>.] - ETA: 0s - loss: 2.4513 - angle_out_loss: 1.5038 - throttle_out_loss: 1.6994 - angle_out_acc: 0.5754 - throttle_out_acc: 0.5577
Epoch 00010: val_loss improved from 2.45861 to 2.42520, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 52ms/step - loss: 2.4534 - angle_out_loss: 1.5047 - throttle_out_loss: 1.7010 - angle_out_acc: 0.5748 - throttle_out_acc: 0.5569 - val_loss: 2.4252 - val_angle_out_loss: 1.4220 - val_throttle_out_loss: 1.7142 - val_angle_out_acc: 0.5820 - val_throttle_out_acc: 0.5449
Epoch 11/100
62/64 [============================>.] - ETA: 0s - loss: 2.4054 - angle_out_loss: 1.4839 - throttle_out_loss: 1.6635 - angle_out_acc: 0.5725 - throttle_out_acc: 0.5601
Epoch 00011: val_loss improved from 2.42520 to 2.36648, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 52ms/step - loss: 2.4062 - angle_out_loss: 1.4828 - throttle_out_loss: 1.6648 - angle_out_acc: 0.5730 - throttle_out_acc: 0.5603 - val_loss: 2.3665 - val_angle_out_loss: 1.3953 - val_throttle_out_loss: 1.6688 - val_angle_out_acc: 0.6045 - val_throttle_out_acc: 0.5698
Epoch 12/100
63/64 [============================>.] - ETA: 0s - loss: 2.3627 - angle_out_loss: 1.4525 - throttle_out_loss: 1.6365 - angle_out_acc: 0.5794 - throttle_out_acc: 0.5682
Epoch 00012: val_loss improved from 2.36648 to 2.31995, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.3616 - angle_out_loss: 1.4535 - throttle_out_loss: 1.6349 - angle_out_acc: 0.5793 - throttle_out_acc: 0.5680 - val_loss: 2.3199 - val_angle_out_loss: 1.3824 - val_throttle_out_loss: 1.6288 - val_angle_out_acc: 0.5928 - val_throttle_out_acc: 0.5635
Epoch 13/100
63/64 [============================>.] - ETA: 0s - loss: 2.3167 - angle_out_loss: 1.4297 - throttle_out_loss: 1.6018 - angle_out_acc: 0.5877 - throttle_out_acc: 0.5725
Epoch 00013: val_loss improved from 2.31995 to 2.31693, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.3165 - angle_out_loss: 1.4279 - throttle_out_loss: 1.6026 - angle_out_acc: 0.5880 - throttle_out_acc: 0.5724 - val_loss: 2.3169 - val_angle_out_loss: 1.3568 - val_throttle_out_loss: 1.6385 - val_angle_out_acc: 0.5928 - val_throttle_out_acc: 0.5605
Epoch 14/100
62/64 [============================>.] - ETA: 0s - loss: 2.2868 - angle_out_loss: 1.4144 - throttle_out_loss: 1.5796 - angle_out_acc: 0.5868 - throttle_out_acc: 0.5766
Epoch 00014: val_loss improved from 2.31693 to 2.24896, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.2894 - angle_out_loss: 1.4154 - throttle_out_loss: 1.5817 - angle_out_acc: 0.5872 - throttle_out_acc: 0.5752 - val_loss: 2.2490 - val_angle_out_loss: 1.3076 - val_throttle_out_loss: 1.5952 - val_angle_out_acc: 0.6035 - val_throttle_out_acc: 0.5737
Epoch 15/100
63/64 [============================>.] - ETA: 0s - loss: 2.2467 - angle_out_loss: 1.3801 - throttle_out_loss: 1.5566 - angle_out_acc: 0.5887 - throttle_out_acc: 0.5745
Epoch 00015: val_loss did not improve from 2.24896
64/64 [==============================] - 3s 50ms/step - loss: 2.2467 - angle_out_loss: 1.3810 - throttle_out_loss: 1.5562 - angle_out_acc: 0.5884 - throttle_out_acc: 0.5742 - val_loss: 2.2712 - val_angle_out_loss: 1.3112 - val_throttle_out_loss: 1.6156 - val_angle_out_acc: 0.6089 - val_throttle_out_acc: 0.5625
Epoch 16/100
63/64 [============================>.] - ETA: 0s - loss: 2.2113 - angle_out_loss: 1.3579 - throttle_out_loss: 1.5323 - angle_out_acc: 0.5946 - throttle_out_acc: 0.5812
Epoch 00016: val_loss improved from 2.24896 to 2.23349, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 50ms/step - loss: 2.2103 - angle_out_loss: 1.3572 - throttle_out_loss: 1.5316 - angle_out_acc: 0.5953 - throttle_out_acc: 0.5813 - val_loss: 2.2335 - val_angle_out_loss: 1.2920 - val_throttle_out_loss: 1.5875 - val_angle_out_acc: 0.6113 - val_throttle_out_acc: 0.5732
Epoch 17/100
62/64 [============================>.] - ETA: 0s - loss: 2.1604 - angle_out_loss: 1.3338 - throttle_out_loss: 1.4935 - angle_out_acc: 0.5997 - throttle_out_acc: 0.5882
Epoch 00017: val_loss improved from 2.23349 to 2.20767, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.1608 - angle_out_loss: 1.3312 - throttle_out_loss: 1.4952 - angle_out_acc: 0.6007 - throttle_out_acc: 0.5875 - val_loss: 2.2077 - val_angle_out_loss: 1.2500 - val_throttle_out_loss: 1.5827 - val_angle_out_acc: 0.6270 - val_throttle_out_acc: 0.5757
Epoch 18/100
62/64 [============================>.] - ETA: 0s - loss: 2.1226 - angle_out_loss: 1.3036 - throttle_out_loss: 1.4708 - angle_out_acc: 0.6040 - throttle_out_acc: 0.5903
Epoch 00018: val_loss improved from 2.20767 to 2.18893, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 50ms/step - loss: 2.1239 - angle_out_loss: 1.3017 - throttle_out_loss: 1.4730 - angle_out_acc: 0.6046 - throttle_out_acc: 0.5903 - val_loss: 2.1889 - val_angle_out_loss: 1.2589 - val_throttle_out_loss: 1.5595 - val_angle_out_acc: 0.6172 - val_throttle_out_acc: 0.5835
Epoch 19/100
62/64 [============================>.] - ETA: 0s - loss: 2.0955 - angle_out_loss: 1.2993 - throttle_out_loss: 1.4459 - angle_out_acc: 0.6119 - throttle_out_acc: 0.6006
Epoch 00019: val_loss improved from 2.18893 to 2.16634, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.1009 - angle_out_loss: 1.3013 - throttle_out_loss: 1.4503 - angle_out_acc: 0.6113 - throttle_out_acc: 0.5996 - val_loss: 2.1663 - val_angle_out_loss: 1.2460 - val_throttle_out_loss: 1.5433 - val_angle_out_acc: 0.6211 - val_throttle_out_acc: 0.5923
Epoch 20/100
63/64 [============================>.] - ETA: 0s - loss: 2.0628 - angle_out_loss: 1.2745 - throttle_out_loss: 1.4256 - angle_out_acc: 0.6136 - throttle_out_acc: 0.6017
Epoch 00020: val_loss did not improve from 2.16634
64/64 [==============================] - 3s 50ms/step - loss: 2.0644 - angle_out_loss: 1.2772 - throttle_out_loss: 1.4259 - angle_out_acc: 0.6121 - throttle_out_acc: 0.6019 - val_loss: 2.1808 - val_angle_out_loss: 1.2118 - val_throttle_out_loss: 1.5749 - val_angle_out_acc: 0.6313 - val_throttle_out_acc: 0.5771
Epoch 21/100
63/64 [============================>.] - ETA: 0s - loss: 2.0410 - angle_out_loss: 1.2611 - throttle_out_loss: 1.4104 - angle_out_acc: 0.6146 - throttle_out_acc: 0.6104
Epoch 00021: val_loss improved from 2.16634 to 2.15919, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 2.0427 - angle_out_loss: 1.2631 - throttle_out_loss: 1.4111 - angle_out_acc: 0.6147 - throttle_out_acc: 0.6101 - val_loss: 2.1592 - val_angle_out_loss: 1.2323 - val_throttle_out_loss: 1.5430 - val_angle_out_acc: 0.6284 - val_throttle_out_acc: 0.5903
Epoch 22/100
63/64 [============================>.] - ETA: 0s - loss: 2.0196 - angle_out_loss: 1.2475 - throttle_out_loss: 1.3958 - angle_out_acc: 0.6190 - throttle_out_acc: 0.6040
Epoch 00022: val_loss improved from 2.15919 to 2.09381, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 50ms/step - loss: 2.0210 - angle_out_loss: 1.2475 - throttle_out_loss: 1.3973 - angle_out_acc: 0.6200 - throttle_out_acc: 0.6041 - val_loss: 2.0938 - val_angle_out_loss: 1.1728 - val_throttle_out_loss: 1.5074 - val_angle_out_acc: 0.6406 - val_throttle_out_acc: 0.5923
Epoch 23/100
63/64 [============================>.] - ETA: 0s - loss: 1.9801 - angle_out_loss: 1.2206 - throttle_out_loss: 1.3698 - angle_out_acc: 0.6231 - throttle_out_acc: 0.6128
Epoch 00023: val_loss improved from 2.09381 to 2.08714, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 1.9815 - angle_out_loss: 1.2210 - throttle_out_loss: 1.3710 - angle_out_acc: 0.6224 - throttle_out_acc: 0.6121 - val_loss: 2.0871 - val_angle_out_loss: 1.1866 - val_throttle_out_loss: 1.4938 - val_angle_out_acc: 0.6401 - val_throttle_out_acc: 0.6104
Epoch 24/100
62/64 [============================>.] - ETA: 0s - loss: 1.9418 - angle_out_loss: 1.2092 - throttle_out_loss: 1.3372 - angle_out_acc: 0.6261 - throttle_out_acc: 0.6196
Epoch 00024: val_loss did not improve from 2.08714
64/64 [==============================] - 3s 50ms/step - loss: 1.9433 - angle_out_loss: 1.2085 - throttle_out_loss: 1.3391 - angle_out_acc: 0.6266 - throttle_out_acc: 0.6180 - val_loss: 2.0993 - val_angle_out_loss: 1.1816 - val_throttle_out_loss: 1.5085 - val_angle_out_acc: 0.6465 - val_throttle_out_acc: 0.5864
Epoch 25/100
62/64 [============================>.] - ETA: 0s - loss: 1.9338 - angle_out_loss: 1.2083 - throttle_out_loss: 1.3297 - angle_out_acc: 0.6255 - throttle_out_acc: 0.6216
Epoch 00025: val_loss improved from 2.08714 to 2.06859, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 50ms/step - loss: 1.9345 - angle_out_loss: 1.2086 - throttle_out_loss: 1.3302 - angle_out_acc: 0.6249 - throttle_out_acc: 0.6223 - val_loss: 2.0686 - val_angle_out_loss: 1.1483 - val_throttle_out_loss: 1.4944 - val_angle_out_acc: 0.6548 - val_throttle_out_acc: 0.5981
Epoch 26/100
62/64 [============================>.] - ETA: 0s - loss: 1.9065 - angle_out_loss: 1.1931 - throttle_out_loss: 1.3100 - angle_out_acc: 0.6347 - throttle_out_acc: 0.6264
Epoch 00026: val_loss improved from 2.06859 to 2.04724, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 50ms/step - loss: 1.9099 - angle_out_loss: 1.1940 - throttle_out_loss: 1.3129 - angle_out_acc: 0.6349 - throttle_out_acc: 0.6260 - val_loss: 2.0472 - val_angle_out_loss: 1.1247 - val_throttle_out_loss: 1.4849 - val_angle_out_acc: 0.6597 - val_throttle_out_acc: 0.6099
Epoch 27/100
63/64 [============================>.] - ETA: 0s - loss: 1.8839 - angle_out_loss: 1.1701 - throttle_out_loss: 1.2988 - angle_out_acc: 0.6394 - throttle_out_acc: 0.6255
Epoch 00027: val_loss did not improve from 2.04724
64/64 [==============================] - 3s 50ms/step - loss: 1.8852 - angle_out_loss: 1.1681 - throttle_out_loss: 1.3012 - angle_out_acc: 0.6398 - throttle_out_acc: 0.6250 - val_loss: 2.0572 - val_angle_out_loss: 1.1412 - val_throttle_out_loss: 1.4866 - val_angle_out_acc: 0.6479 - val_throttle_out_acc: 0.5933
Epoch 28/100
63/64 [============================>.] - ETA: 0s - loss: 1.8520 - angle_out_loss: 1.1541 - throttle_out_loss: 1.2749 - angle_out_acc: 0.6404 - throttle_out_acc: 0.6341
Epoch 00028: val_loss did not improve from 2.04724
64/64 [==============================] - 3s 50ms/step - loss: 1.8504 - angle_out_loss: 1.1530 - throttle_out_loss: 1.2740 - angle_out_acc: 0.6406 - throttle_out_acc: 0.6342 - val_loss: 2.0524 - val_angle_out_loss: 1.1295 - val_throttle_out_loss: 1.4877 - val_angle_out_acc: 0.6543 - val_throttle_out_acc: 0.5962
Epoch 29/100
62/64 [============================>.] - ETA: 0s - loss: 1.8219 - angle_out_loss: 1.1461 - throttle_out_loss: 1.2488 - angle_out_acc: 0.6444 - throttle_out_acc: 0.6342
Epoch 00029: val_loss did not improve from 2.04724
64/64 [==============================] - 3s 50ms/step - loss: 1.8202 - angle_out_loss: 1.1419 - throttle_out_loss: 1.2493 - angle_out_acc: 0.6456 - throttle_out_acc: 0.6344 - val_loss: 2.0724 - val_angle_out_loss: 1.1363 - val_throttle_out_loss: 1.5042 - val_angle_out_acc: 0.6602 - val_throttle_out_acc: 0.6060
Epoch 30/100
63/64 [============================>.] - ETA: 0s - loss: 1.8018 - angle_out_loss: 1.1369 - throttle_out_loss: 1.2334 - angle_out_acc: 0.6469 - throttle_out_acc: 0.6393
Epoch 00030: val_loss improved from 2.04724 to 2.03566, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 1.8012 - angle_out_loss: 1.1339 - throttle_out_loss: 1.2343 - angle_out_acc: 0.6475 - throttle_out_acc: 0.6389 - val_loss: 2.0357 - val_angle_out_loss: 1.1356 - val_throttle_out_loss: 1.4679 - val_angle_out_acc: 0.6475 - val_throttle_out_acc: 0.6104
Epoch 31/100
62/64 [============================>.] - ETA: 0s - loss: 1.7853 - angle_out_loss: 1.1236 - throttle_out_loss: 1.2235 - angle_out_acc: 0.6479 - throttle_out_acc: 0.6462
Epoch 00031: val_loss improved from 2.03566 to 2.02520, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 1.7898 - angle_out_loss: 1.1253 - throttle_out_loss: 1.2271 - angle_out_acc: 0.6472 - throttle_out_acc: 0.6450 - val_loss: 2.0252 - val_angle_out_loss: 1.1151 - val_throttle_out_loss: 1.4677 - val_angle_out_acc: 0.6616 - val_throttle_out_acc: 0.6143
Epoch 32/100
62/64 [============================>.] - ETA: 0s - loss: 1.7563 - angle_out_loss: 1.1123 - throttle_out_loss: 1.2002 - angle_out_acc: 0.6503 - throttle_out_acc: 0.6525
Epoch 00032: val_loss did not improve from 2.02520
64/64 [==============================] - 3s 50ms/step - loss: 1.7569 - angle_out_loss: 1.1140 - throttle_out_loss: 1.1998 - angle_out_acc: 0.6493 - throttle_out_acc: 0.6523 - val_loss: 2.0255 - val_angle_out_loss: 1.1203 - val_throttle_out_loss: 1.4654 - val_angle_out_acc: 0.6572 - val_throttle_out_acc: 0.6123
Epoch 33/100
63/64 [============================>.] - ETA: 0s - loss: 1.7554 - angle_out_loss: 1.1136 - throttle_out_loss: 1.1985 - angle_out_acc: 0.6458 - throttle_out_acc: 0.6498
Epoch 00033: val_loss did not improve from 2.02520
64/64 [==============================] - 3s 50ms/step - loss: 1.7560 - angle_out_loss: 1.1128 - throttle_out_loss: 1.1996 - angle_out_acc: 0.6458 - throttle_out_acc: 0.6493 - val_loss: 2.0505 - val_angle_out_loss: 1.1340 - val_throttle_out_loss: 1.4835 - val_angle_out_acc: 0.6592 - val_throttle_out_acc: 0.6099
Epoch 34/100
63/64 [============================>.] - ETA: 0s - loss: 1.7325 - angle_out_loss: 1.1067 - throttle_out_loss: 1.1791 - angle_out_acc: 0.6527 - throttle_out_acc: 0.6545
Epoch 00034: val_loss improved from 2.02520 to 1.99127, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 1.7275 - angle_out_loss: 1.1035 - throttle_out_loss: 1.1757 - angle_out_acc: 0.6533 - throttle_out_acc: 0.6553 - val_loss: 1.9913 - val_angle_out_loss: 1.0866 - val_throttle_out_loss: 1.4480 - val_angle_out_acc: 0.6636 - val_throttle_out_acc: 0.6226
Epoch 35/100
63/64 [============================>.] - ETA: 0s - loss: 1.7103 - angle_out_loss: 1.0837 - throttle_out_loss: 1.1684 - angle_out_acc: 0.6601 - throttle_out_acc: 0.6572
Epoch 00035: val_loss did not improve from 1.99127
64/64 [==============================] - 3s 50ms/step - loss: 1.7076 - angle_out_loss: 1.0810 - throttle_out_loss: 1.1672 - angle_out_acc: 0.6608 - throttle_out_acc: 0.6576 - val_loss: 1.9994 - val_angle_out_loss: 1.0877 - val_throttle_out_loss: 1.4555 - val_angle_out_acc: 0.6660 - val_throttle_out_acc: 0.6099
Epoch 36/100
63/64 [============================>.] - ETA: 0s - loss: 1.6924 - angle_out_loss: 1.0825 - throttle_out_loss: 1.1511 - angle_out_acc: 0.6592 - throttle_out_acc: 0.6610
Epoch 00036: val_loss did not improve from 1.99127
64/64 [==============================] - 3s 50ms/step - loss: 1.6950 - angle_out_loss: 1.0821 - throttle_out_loss: 1.1539 - angle_out_acc: 0.6593 - throttle_out_acc: 0.6594 - val_loss: 2.0371 - val_angle_out_loss: 1.1129 - val_throttle_out_loss: 1.4807 - val_angle_out_acc: 0.6567 - val_throttle_out_acc: 0.6104
Epoch 37/100
63/64 [============================>.] - ETA: 0s - loss: 1.6769 - angle_out_loss: 1.0704 - throttle_out_loss: 1.1417 - angle_out_acc: 0.6638 - throttle_out_acc: 0.6616
Epoch 00037: val_loss did not improve from 1.99127
64/64 [==============================] - 3s 50ms/step - loss: 1.6785 - angle_out_loss: 1.0716 - throttle_out_loss: 1.1427 - angle_out_acc: 0.6636 - throttle_out_acc: 0.6611 - val_loss: 2.0072 - val_angle_out_loss: 1.0874 - val_throttle_out_loss: 1.4635 - val_angle_out_acc: 0.6646 - val_throttle_out_acc: 0.6118
Epoch 38/100
63/64 [============================>.] - ETA: 0s - loss: 1.6569 - angle_out_loss: 1.0749 - throttle_out_loss: 1.1194 - angle_out_acc: 0.6618 - throttle_out_acc: 0.6710
Epoch 00038: val_loss did not improve from 1.99127
64/64 [==============================] - 3s 50ms/step - loss: 1.6566 - angle_out_loss: 1.0776 - throttle_out_loss: 1.1178 - angle_out_acc: 0.6605 - throttle_out_acc: 0.6713 - val_loss: 2.0223 - val_angle_out_loss: 1.0740 - val_throttle_out_loss: 1.4853 - val_angle_out_acc: 0.6782 - val_throttle_out_acc: 0.6196
Epoch 39/100
63/64 [============================>.] - ETA: 0s - loss: 1.6337 - angle_out_loss: 1.0643 - throttle_out_loss: 1.1015 - angle_out_acc: 0.6638 - throttle_out_acc: 0.6716
Epoch 00039: val_loss improved from 1.99127 to 1.94174, saving model to /content/mycar/models/mypilot.h5
64/64 [==============================] - 3s 51ms/step - loss: 1.6348 - angle_out_loss: 1.0660 - throttle_out_loss: 1.1018 - angle_out_acc: 0.6639 - throttle_out_acc: 0.6718 - val_loss: 1.9417 - val_angle_out_loss: 1.1158 - val_throttle_out_loss: 1.3838 - val_angle_out_acc: 0.6641 - val_throttle_out_acc: 0.6177
Epoch 40/100
63/64 [============================>.] - ETA: 0s - loss: 1.6047 - angle_out_loss: 1.0482 - throttle_out_loss: 1.0806 - angle_out_acc: 0.6632 - throttle_out_acc: 0.6727
Epoch 00040: val_loss did not improve from 1.94174
64/64 [==============================] - 3s 50ms/step - loss: 1.6059 - angle_out_loss: 1.0478 - throttle_out_loss: 1.0820 - angle_out_acc: 0.6638 - throttle_out_acc: 0.6726 - val_loss: 2.1267 - val_angle_out_loss: 1.1378 - val_throttle_out_loss: 1.5578 - val_angle_out_acc: 0.6611 - val_throttle_out_acc: 0.5977
Epoch 41/100
63/64 [============================>.] - ETA: 0s - loss: 1.6027 - angle_out_loss: 1.0421 - throttle_out_loss: 1.0816 - angle_out_acc: 0.6705 - throttle_out_acc: 0.6760
Epoch 00041: val_loss did not improve from 1.94174
64/64 [==============================] - 3s 50ms/step - loss: 1.6017 - angle_out_loss: 1.0461 - throttle_out_loss: 1.0786 - angle_out_acc: 0.6697 - throttle_out_acc: 0.6770 - val_loss: 2.0650 - val_angle_out_loss: 1.0897 - val_throttle_out_loss: 1.5202 - val_angle_out_acc: 0.6782 - val_throttle_out_acc: 0.6074
Epoch 42/100
62/64 [============================>.] - ETA: 0s - loss: 1.6009 - angle_out_loss: 1.0586 - throttle_out_loss: 1.0716 - angle_out_acc: 0.6682 - throttle_out_acc: 0.6746
Epoch 00042: val_loss did not improve from 1.94174
64/64 [==============================] - 3s 51ms/step - loss: 1.5973 - angle_out_loss: 1.0587 - throttle_out_loss: 1.0679 - angle_out_acc: 0.6682 - throttle_out_acc: 0.6753 - val_loss: 2.0174 - val_angle_out_loss: 1.0967 - val_throttle_out_loss: 1.4690 - val_angle_out_acc: 0.6685 - val_throttle_out_acc: 0.6221
Epoch 43/100
63/64 [============================>.] - ETA: 0s - loss: 1.5671 - angle_out_loss: 1.0455 - throttle_out_loss: 1.0444 - angle_out_acc: 0.6656 - throttle_out_acc: 0.6837
Epoch 00043: val_loss did not improve from 1.94174
64/64 [==============================] - 3s 50ms/step - loss: 1.5656 - angle_out_loss: 1.0453 - throttle_out_loss: 1.0430 - angle_out_acc: 0.6660 - throttle_out_acc: 0.6840 - val_loss: 2.0173 - val_angle_out_loss: 1.1034 - val_throttle_out_loss: 1.4656 - val_angle_out_acc: 0.6606 - val_throttle_out_acc: 0.6128
Epoch 44/100
63/64 [============================>.] - ETA: 0s - loss: 1.5590 - angle_out_loss: 1.0230 - throttle_out_loss: 1.0475 - angle_out_acc: 0.6747 - throttle_out_acc: 0.6850
Epoch 00044: val_loss did not improve from 1.94174
64/64 [==============================] - 3s 50ms/step - loss: 1.5619 - angle_out_loss: 1.0276 - throttle_out_loss: 1.0481 - angle_out_acc: 0.6736 - throttle_out_acc: 0.6849 - val_loss: 2.0355 - val_angle_out_loss: 1.0995 - val_throttle_out_loss: 1.4858 - val_angle_out_acc: 0.6763 - val_throttle_out_acc: 0.6069
Epoch 00044: early stopping
Training completed in 0:02:31.
に変更した。DEFAULT_MODEL_TYPE="categorical"
(env) pi@donkeypi:~/mycar $ python manage.py drive --model ~/mycar/models/mypilot.h5
using donkey v3.0.2 ...
loading config file: /home/pi/mycar/config.py
loading personal config over-rides
config loaded
cfg.CAMERA_TYPE PICAM
PiCamera loaded.. .warming camera
Adding part PiCamera.
Starting Donkey Server...
Adding part LocalWebController.
Adding part ThrottleFilter.
Adding part PilotCondition.
Adding part RecordTracker.
"get_model_by_type" model Type is: categorical
loading model /home/pi/mycar/models/mypilot.h5
finished loading in 12.697422981262207 sec.
Adding part FileWatcher.
Adding part FileWatcher.
Adding part DelayedTrigger.
Adding part TriggeredCallback.
Adding part KerasCategorical.
Adding part DriveMode.
Adding part AiLaunch.
Adding part AiRunCondition.
Init ESC
Adding part PWMSteering.
Adding part PWMThrottle.
Tub does NOT exist. Creating new tub...
New tub created at: /home/pi/mycar/data/tub_8_19-07-30
Adding part TubWriter.
You can now go to <your pi ip address>:8887 to drive your car.
Starting vehicle...
8887
/usr/lib/python3/dist-packages/picamera/encoders.py:544: PiCameraResolutionRounded: frame size rounded up from 160x120 to 160x128
width, height, fwidth, fheight)))
WARNING: Logging before flag parsing goes to stderr.
W0730 20:43:33.409733 1518576752 web.py:2246] 404 GET /favicon.ico (192.168.3.7) 11.80ms
^CShutting down vehicle and its parts...
stoping PiCamera
Part Profile Summary: (times in ms)
+--------------------+------------------+-------+--------------+
| part | max | min | avg |
+--------------------+------------------+-------+--------------+
| PWMThrottle | 20.27 | 1.46 | 2.60 |
| FileWatcher | 7.28 | 0.08 | 0.23 |
| PWMSteering | 1564516404941.14 | 1.55 | 128660891.52 |
| LocalWebController | 0.63 | 0.04 | 0.11 |
| DriveMode | 0.80 | 0.04 | 0.08 |
| AiRunCondition | 0.46 | 0.02 | 0.05 |
| ThrottleFilter | 0.49 | 0.03 | 0.06 |
| PilotCondition | 0.26 | 0.02 | 0.05 |
| RecordTracker | 0.30 | 0.02 | 0.05 |
| KerasCategorical | 2200.69 | 42.03 | 72.93 |
| TriggeredCallback | 0.29 | 0.02 | 0.03 |
| PiCamera | 0.50 | 0.02 | 0.07 |
| AiLaunch | 0.76 | 0.03 | 0.06 |
| FileWatcher | 1.64 | 0.04 | 0.08 |
| DelayedTrigger | 0.18 | 0.02 | 0.04 |
+--------------------+------------------+-------+--------------+
{"user/mode": "user", "user/angle": 0.0, "user/throttle": 3.051850947599719e-05, "cam/image_array": "1_cam-image_array_.jpg", "milliseconds": 323}
/dts-v1/;/plugin/;
/ {
fragment@0 {
target-path = "/amba_pl@0";
#address-celss = <2>;
#size-cells = <2>;
__overlay__ {
v4l2 {
compatible = "fixstars,zynq-v4l2-1.0";
#interrupt-cells = <0x3>;
device-name="v4l2";
interrupt-parent = <&gic>;
interrupts = <0x0 0x59 0x4>;
};
axi_vdma_uio {
compatible = "generic-uio";
reg = <0x0 0xA0010000 0x0 0x1000>;
};
display_dmar_axis_vga_uio {
compatible = "generic-uio";
reg = <0x0 0xA0030000 0x0 0x10000>;
};
disp_gpio_uio {
compatible = "generic-uio";
reg = <0x0 0xA0011000 0x0 0x1000>;
};
gpio_uio {
compatible = "generic-uio";
reg = <0x0 0xA0012000 0x0 0x1000>;
};
};
};
};
/dts-v1/;
/ {
fragment@0 {
target-path = "/fpga-full";
__overlay__ {
firmware-name = "fpga_dp.bin";
};
};
};
// pcam5c_disp_dp.cpp
// 2019/07/24 by marsee
//
// This software converts the left and right of the camera image to BMP file.
// -b : bmp file name
// -n : Start File Number
// -h : help
#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <string.h>
#define PIXEL_NUM_OF_BYTES 4
#define NUMBER_OF_WRITE_FRAMES 3
#define SVGA_HORIZONTAL_PIXELS 800
#define SVGA_VERTICAL_LINES 600
#define SVGA_ALL_DISP_ADDRESS (SVGA_HORIZONTAL_PIXELS * SVGA_VERTICAL_LINES * PIXEL_NUM_OF_BYTES)
#define SVGA_3_PICTURES (SVGA_ALL_DISP_ADDRESS * NUMBER_OF_WRITE_FRAMES)
#define XGA_HORIZONTAL_PIXELS 1024
#define XGA_VERTICAL_LINES 768
#define XGA_ALL_DISP_ADDRESS (XGA_HORIZONTAL_PIXELS * XGA_VERTICAL_LINES * PIXEL_NUM_OF_BYTES)
#define XGA_3_PICTURES (XGA_ALL_DISP_ADDRESS * NUMBER_OF_WRITE_FRAMES)
#define HD_HORIZONTAL_PIXELS 1920
#define HD_VERTICAL_LINES 1080
#define HD_ALL_DISP_ADDRESS (HD_HORIZONTAL_PIXELS * HD_VERTICAL_LINES * PIXEL_NUM_OF_BYTES)
#define HD_3_PICTURES (HD_ALL_DISP_ADDRESS * NUMBER_OF_WRITE_FRAMES)
int main(int argc, char *argv[]){
int opt;
int c, help_flag=0;
char bmp_fn[256] = "bmp_file";
char attr[1024];
unsigned long phys_addr;
int file_no = -1;
int fd1, fd2, fd3, fd4;
volatile unsigned int *axi_vdma_uio, *display_dmar_axis_vga_uio, *disp_gpio_uio, *gpio_uio;
int active_frame;
int resolution;
int all_disp_addr;
uint32_t row, col;
int disp_active_frame=0;
resolution = 1; // XGA
while ((opt=getopt(argc, argv, "b:n:h:r:")) != -1){
switch (opt){
case 'b':
strcpy(bmp_fn, optarg);
break;
case 'n':
file_no = atoi(optarg);
printf("file_no = %d\n", file_no+1);
break;
case 'r':
resolution = atoi(optarg);
break;
case 'h':
help_flag = 1;
break;
}
}
if(resolution == 0){
printf("SVGA\n");
} else if(resolution == 1){
printf("XGA\n");
} else {
printf("HD\n");
}
// all_disp_addr
switch(resolution){
case 0 :
all_disp_addr = SVGA_ALL_DISP_ADDRESS;
row = SVGA_VERTICAL_LINES; col = SVGA_HORIZONTAL_PIXELS;
break;
case 1 :
all_disp_addr = XGA_ALL_DISP_ADDRESS;
row = XGA_VERTICAL_LINES; col = XGA_HORIZONTAL_PIXELS;
break;
default : // 2
all_disp_addr = HD_ALL_DISP_ADDRESS;
row = HD_VERTICAL_LINES; col = HD_HORIZONTAL_PIXELS;
break;
}
// axi_vdma_uio IP
fd1 = open("/dev/uio1", O_RDWR|O_SYNC); // Read/Write, The chache is disable
if (fd1 < 1){
fprintf(stderr, "/dev/uio1 (axi_vdma_uio) open error\n");
exit(-1);
}
axi_vdma_uio = (volatile unsigned *)mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, fd1, 0);
if (!axi_vdma_uio){
fprintf(stderr, "axi_vdma_uio mmap error\n");
exit(-1);
}
// display_dmar_axis_vga_uio IP
fd2 = open("/dev/uio2", O_RDWR|O_SYNC); // Read/Write, The chache is disable
if (fd2 < 1){
fprintf(stderr, "/dev/uio2 (display_dmar_axis_vga_uio) open error\n");
exit(-1);
}
display_dmar_axis_vga_uio = (volatile unsigned *)mmap(NULL, 0x10000, PROT_READ|PROT_WRITE, MAP_SHARED, fd2, 0);
if (!display_dmar_axis_vga_uio){
fprintf(stderr, "display_dmar_axis_vga_uio mmap error\n");
exit(-1);
}
// disp_gpio_uio IP
fd3 = open("/dev/uio3", O_RDWR|O_SYNC); // Read/Write, The chache is disable
if (fd3 < 1){
fprintf(stderr, "/dev/uio3 (disp_gpio_uio) open error\n");
exit(-1);
}
disp_gpio_uio = (volatile unsigned *)mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, fd3, 0);
if (!disp_gpio_uio){
fprintf(stderr, "disp_gpio_uio mmap error\n");
exit(-1);
}
// gpio_uio IP
fd4 = open("/dev/uio4", O_RDWR|O_SYNC); // Read/Write, The chache is disable
if (fd4 < 1){
fprintf(stderr, "/dev/uio4 (gpio_uio) open error\n");
exit(-1);
}
gpio_uio = (volatile unsigned *)mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, fd4, 0);
if (!gpio_uio){
fprintf(stderr, "gpio_uio mmap error\n");
exit(-1);
}
uint32_t framebuf0, framebuf1, framebuf2;
framebuf0 = axi_vdma_uio[43]; // 0xac
framebuf1 = axi_vdma_uio[44]; // 0xb0
framebuf2 = axi_vdma_uio[45]; // 0xb4
printf("framebuf0 = %x; framebuf1 = %x; framebuf2 = %x\n", framebuf0, framebuf1, framebuf2);
display_dmar_axis_vga_uio[4] = framebuf0; // 0x10, fb0_V
display_dmar_axis_vga_uio[6] = framebuf1; // 0x18, fb1_V
display_dmar_axis_vga_uio[8] = framebuf2; // 0x20, fb2_V
display_dmar_axis_vga_uio[10] = row; // 0x28, row_V
display_dmar_axis_vga_uio[12] = col; // 0x30, col_V
disp_gpio_uio[0] = 1; // disp_dmar_axis_vga start(init_done = 1)
c = getc(stdin);
while(c != 'q'){
switch ((char)c) {
case 'f':
printf("active frame = %d\n", gpio_uio[0]);
break;
}
c = getc(stdin);
}
return(0);
}
// axis2video_out.v
// 2019/01/14 by marsee
// 2019/01/23 : Fixed IP start state machine bug
// 2019/07/25 : Added (* ASYNC_REG = "TRUE" *)
//
`default_nettype none
module axis2video_out
(
// Clock and Reset
input wire disp_clk,
input wire axi_clk,
input wire axi_rst_n,
input wire init_done,
// AXI4-Stream
input wire [31:0] axis_tdata,
input wire axis_tvalid,
output wire axis_tready,
input wire [3:0] axis_tkeep,
input wire [3:0] axis_tstrb,
input wire axis_tuser,
input wire axis_tlast,
input wire axis_tid,
input wire axis_tdest,
// IP
output reg ip_start,
input wire ip_done,
// video in
input wire de_in,
input wire vsync_in,
input wire hsync_in,
// video_out
output wire [35:0] disp_pixel,
output wire de_out,
output wire vsync_out,
output wire hsync_out
);
parameter IDLE_START = 1'b0,
IP_START_1 = 1'b1;
reg reset_disp_2b = 1'b1, reset_disp_1b = 1'b1;
wire reset_disp;
reg fifo_reset_axi_2b = 1'b0, fifo_reset_axi_1b = 1'b0;
wire fifo_reset_axi;
reg fifo_reset_disp_2b = 1'b0, fifo_reset_disp_1b = 1'b0;
wire fifo_reset_disp;
reg de_1d, vsync_1d, hsync_1d;
(* ASYNC_REG = "TRUE" *) reg vsync_axi_1b, vsync_axi_2b;
wire vsync_axi;
(* ASYNC_REG = "TRUE" *) reg vsync_axi_1d, vsync_axi_2d;
reg cs_start;
wire pfifo_empty, pfifo_full;
wire [33:0] pfifo_dout;
reg vsync_rising_edge_axi, vsync_falling_edge_axi;
(* ASYNC_REG = "TRUE" *) reg init_done_axi_1d, axi_rst_n_axi_1d;
(* ASYNC_REG = "TRUE" *) reg init_done_axi_2d, axi_rst_n_axi_2d;
(* ASYNC_REG = "TRUE" *) reg init_done_disp_1d, axi_rst_n_disp_1d;
(* ASYNC_REG = "TRUE" *) reg init_done_disp_2d, axi_rst_n_disp_2d;
always @(posedge disp_clk) begin
if(reset_disp) begin
de_1d <= 1'b0;
vsync_1d <= 1'b0;
hsync_1d <= 1'b0;
end else begin
de_1d <= de_in;
vsync_1d <= vsync_in;
hsync_1d <= hsync_in;
end
end
always @(posedge axi_clk) begin
init_done_axi_1d <= init_done;
init_done_axi_2d <= init_done_axi_1d;
axi_rst_n_axi_1d <= axi_rst_n;
axi_rst_n_axi_2d <= axi_rst_n_axi_1d;
end
always @(posedge disp_clk) begin
init_done_disp_1d <= init_done;
init_done_disp_2d <= init_done_disp_1d;
axi_rst_n_disp_1d <= axi_rst_n;
axi_rst_n_disp_2d <= axi_rst_n_disp_1d;
end
// reset signals
always @(posedge axi_clk) begin
fifo_reset_axi_2b <= ~init_done_axi_2d | ~axi_rst_n_axi_2d | vsync_axi;
fifo_reset_axi_1b <= fifo_reset_axi_2b;
end
assign fifo_reset_axi = fifo_reset_axi_1b;
always @(posedge disp_clk) begin
fifo_reset_disp_2b <= ~init_done_disp_2d | ~axi_rst_n_disp_2d | vsync_1d;
fifo_reset_disp_1b <= fifo_reset_disp_2b;
end
assign fifo_reset_disp = fifo_reset_disp_1b;
always @(posedge disp_clk) begin
reset_disp_2b <= ~init_done_disp_2d | ~axi_rst_n_disp_2d;
reset_disp_1b <= reset_disp_2b;
end
assign reset_disp = reset_disp_1b;
// vsync_rising_edge, vsync_falling_edge
always @(posedge axi_clk) begin
if (!axi_rst_n) begin
vsync_axi_2b <= 1'b1;
vsync_axi_1b <= 1'b1;
end else begin
vsync_axi_2b <= vsync_1d;
vsync_axi_1b <= vsync_axi_2b;
end
end
assign vsync_axi = vsync_axi_1b;
always @(posedge axi_clk) begin
if (!axi_rst_n) begin
vsync_axi_1d <= 1'b1;
vsync_axi_2d <= 1'b1;
end else begin
vsync_axi_1d <= vsync_axi;
vsync_axi_2d <= vsync_axi_1d;
end
end
always @(posedge axi_clk) begin
if (!axi_rst_n) begin
vsync_rising_edge_axi = 1'b0;
vsync_falling_edge_axi = 1'b0;
end else begin
vsync_rising_edge_axi <= ~vsync_axi_2d & vsync_axi_1d;
vsync_falling_edge_axi <= vsync_axi_2d & ~vsync_axi_1d;
end
end
// IP start State Machine
always @(posedge axi_clk) begin
if (!axi_rst_n) begin
cs_start <= IDLE_START;
ip_start <= 1'b0;
end else begin
case (cs_start)
IDLE_START : begin
ip_start <= 1'b0;
if (vsync_falling_edge_axi) begin
cs_start <= IP_START_1;
ip_start <= 1'b1;
end
end
IP_START_1 : begin
if (ip_done) begin
cs_start <= IDLE_START;
ip_start <= 1'b0;
end
end
endcase
end
end
// data width 34 bits, 512 depth
pixel_fifo pixel_fifo_i (
.wr_rst(fifo_reset_axi),
.wr_clk(axi_clk),
.rd_rst(fifo_reset_disp),
.rd_clk(disp_clk),
.din({axis_tuser, axis_tlast, axis_tdata}),
.dout(pfifo_dout),
.wr_en(~pfifo_full & axis_tvalid),
.full(pfifo_full),
.rd_en(de_1d),
.empty(pfifo_empty)
);
assign axis_tready = ~pfifo_full;
assign disp_pixel = {pfifo_dout[7:0], 4'd0, pfifo_dout[23:16], 4'd0, pfifo_dout[15:8], 4'd0}; //BRG
assign de_out = de_1d;
assign vsync_out = vsync_1d;
assign hsync_out = hsync_1d;
endmodule
`default_nettype wire
ASYNC_REG では、 タイミング違反が発生しても最後の既知の値を出力するようにレジスタが変更されます。
/dts-v1/;/plugin/;
/ {
fragment@0 {
target-path = "/amba_pl@0";
#address-celss = <2>;
#size-cells = <2>;
__overlay__ {
v4l2 {
compatible = "fixstars,zynq-v4l2-1.0";
#interrupt-cells = <0x3>;
device-name="v4l2";
interrupt-parent = <&gic>;
interrupts = <0x0 0x59 0x4>;
};
axi_vdma_uio {
compatible = "generic-uio";
reg = <0x0 0xA0010000 0x0 0x1000>;
};
display_dmar_axis_vga_uio {
compatible = "generic-uio";
reg = <0x0 0xA0030000 0x0 0x10000>;
};
disp_gpio_uio {
compatible = "generic-uio";
reg = <0x0 0xA0011000 0x0 0x1000>;
};
gpio_uio {
compatible = "generic-uio";
reg = <0x0 0xA0012000 0x0 0x1000>;
};
};
};
};
// memread.c
// 2019/07/21 : by marsee
// I referred to http://independence-sys.net/main/?p=2209
//
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#define BLOCK_SIZE 4096
volatile uint32_t *reg;
int main(int argc, char **argv){
int fd;
void *memp;
uint32_t phy_addr;
uint32_t phy_addr_base;
uint32_t addr, addr2;
uint32_t write_data;
if (argc != 2){
fprintf(stderr, "Usage : ./memread <address(hex)>\n");
exit(-1);
}
fd = open("/dev/mem", O_RDWR | O_SYNC);
if (fd == -1){
fprintf(stderr, "/dev/mem open error\n");
exit(-1);
}
sscanf(argv[1], "%x", &addr);
phy_addr = (uint32_t)addr;
phy_addr_base = phy_addr & 0xfffff000; // 4k byte boundary
memp = mmap(NULL, BLOCK_SIZE,
PROT_READ | PROT_WRITE, MAP_SHARED,
fd, phy_addr_base );
if ((int64_t)memp == -1){
fprintf(stderr,"/dev/mem map error\n");
exit(-1);
}
close(fd);
reg = (uint32_t *)memp;
int index = (phy_addr & 0xfff)/sizeof(uint32_t);
printf("%0.8x\n", reg[index]);
munmap((void *)memp, BLOCK_SIZE);
return(0);
}
all:
{
[destination_device = pl] design_1_wrapper.bit
}
// disp_dmar_axis_vga.cpp
// 2019/07/11 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int vga_dmar_axis(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<2> active_frame);
int axis_expand(AXI_STREAM &ins, AXI_STREAM &outs, int row, int col);
int disp_dmar_axis_vga(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<16> row, ap_uint<16> col, ap_uint<2> &active_frame){
#pragma HLS DATAFLOW
#pragma HLS INTERFACE ap_ctrl_hs port=return
#pragma HLS INTERFACE ap_none port=active_frame
#pragma HLS INTERFACE s_axilite port=row
#pragma HLS INTERFACE s_axilite port=col
#pragma HLS INTERFACE axis register both port=outs
#pragma HLS INTERFACE m_axi depth=307200 port=fb2 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb1 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb0 offset=slave
AXI_STREAM dmad;
#pragma HLS STREAM variable=dmad depth=64 dim=1
vga_dmar_axis(fb0, fb1, fb2, dmad, active_frame);
axis_expand(dmad, outs, row, col);
return(0);
}
// vga_dmar_axis.cpp
// 2019/07/13 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height);
int vga_dmar_axis(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<2> active_frame){
AP_AXIU32 pix;
if (active_frame == (ap_uint<2>)0)
disp_dmar_fb2(fb2, outs, VGA_WIDTH, VGA_HEIGHT);
else if (active_frame == (ap_uint<2>)1)
disp_dmar_fb0(fb0, outs, VGA_WIDTH, VGA_HEIGHT);
else
disp_dmar_fb1(fb1, outs, VGA_WIDTH, VGA_HEIGHT);
return(0);
}
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
LOOP_Y0: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
LOOP_X0: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb0[(y*max_width)+x];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
LOOP_Y1: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
LOOP_X1: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb1[(y*max_width)+x];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
LOOP_Y2: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
LOOP_X2: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb2[(y*max_width)+x];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
// axis_expand.cpp
// 2019/07/13 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int axis_expand(AXI_STREAM &ins, AXI_STREAM &outs, int row, int col){
AP_AXIU32 pix;
int x_padding = (col - VGA_WIDTH)/2;
int y_padding = (row - VGA_HEIGHT)/2;
for(int y=0; y<row; y++){
#pragma HLS LOOP_TRIPCOUNT min=600 max=1080 avg=768
for(int x=0; x<col; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
if (y < y_padding || y >= y_padding+VGA_HEIGHT){
pix.data = 0;
pix.dest = 0;
pix.id = 0;
pix.keep = 0;
pix.strb = 0;
}else if (x < x_padding || x >= x_padding+VGA_WIDTH){
pix.data = 0;
pix.dest = 0;
pix.id = 0;
pix.keep = 0;
pix.strb = 0;
}else{
ins >> pix;
}
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == col-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
// disp_dmar_axis_vga.h
// 2019/07/11 by marsee
//
#ifndef __DISP_DMAR_AXIS_VGA_H__
#define __DISP_DMAR_AXIS_VGA_H__
#include "ap_axi_sdata.h"
#include "hls_video.h"
#define VGA_WIDTH 640
#define VGA_HEIGHT 480
#define SVGA_WIDTH 800
#define SVGA_HEIGHT 600
#define XGA_WIDTH 1024
#define XGA_HEIGHT 768
#define HD_WIDTH 1920
#define HD_HEIGHT 1080
#define RESO_SVGA 0
#define RESO_XGA 1
#define RESO_HD 2
typedef hls::stream<ap_axiu<32,1,1,1> > AXI_STREAM;
typedef ap_axiu<32,1,1,1> AP_AXIU32;
typedef hls::Scalar<3, unsigned char> RGB_PIXEL;
typedef hls::Mat<HD_HEIGHT, HD_WIDTH, HLS_8UC3> RGB_IMAGE;
typedef hls::Mat<HD_HEIGHT, HD_WIDTH, HLS_8UC1> GRAY_IMAGE;
#endif
// disp_dmar_axis_vga.cpp
// 2019/07/11 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int disp_dmar_stream(AXI_STREAM &ins, AXI_STREAM &outs, int max_width, int max_height);
int fb0_hls_stream(volatile ap_int<32> *fb, AXI_STREAM &outs);
int fb1_hls_stream(volatile ap_int<32> *fb, AXI_STREAM &outs);
int fb2_hls_stream(volatile ap_int<32> *fb, AXI_STREAM &outs);
int disp_dmar_axis_vga(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<16> row, ap_uint<16> col, ap_uint<2> active_frame){
#pragma HLS INTERFACE s_axilite port=col
#pragma HLS INTERFACE s_axilite port=row
#pragma HLS DATAFLOW
#pragma HLS INTERFACE ap_ctrl_hs port=return
#pragma HLS INTERFACE ap_none port=active_frame
#pragma HLS INTERFACE axis register both port=outs
#pragma HLS INTERFACE m_axi depth=307200 port=fb2 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb1 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb0 offset=slave
AP_AXIU32 pix;
AXI_STREAM indata0;
AXI_STREAM indata1;
AXI_STREAM indata2;
#pragma HLS STREAM variable=indata0 depth=64 dim=1
#pragma HLS STREAM variable=indata1 depth=64 dim=1
#pragma HLS STREAM variable=indata2 depth=64 dim=1
if (active_frame == (ap_uint<2>)0){
fb2_hls_stream(fb2, indata0);
disp_dmar_stream(indata0, outs, col, row);
}else if (active_frame == (ap_uint<2>)1){
fb0_hls_stream(fb0, indata1);
disp_dmar_stream(indata1, outs, col, row);
}else{
fb1_hls_stream(fb1, indata2);
disp_dmar_stream(indata2, outs, col, row);
}
return(0);
}
int disp_dmar_stream(AXI_STREAM &ins, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
AP_AXIU32 dmad;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
LOOP_Y0: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=600 max=1080 avg=768
LOOP_X0: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
if (y < y_padding || y >= y_padding+VGA_HEIGHT)
pix.data = 0;
else if (x < x_padding || x >= x_padding+VGA_WIDTH)
pix.data = 0;
else{
ins >> dmad;
pix.data = dmad.data;
}
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
int fb0_hls_stream(volatile ap_int<32> *fb0, AXI_STREAM &outs){
AP_AXIU32 dmad;
LOOP_Y1: for(int y=0; y<VGA_HEIGHT; y++){
LOOP_X1: for(int x=0; x<VGA_WIDTH; x++){
#pragma HLS PIPELINE II=1
dmad.data = fb0[(y*VGA_WIDTH)+x];
outs << dmad;
}
}
return(0);
}
int fb1_hls_stream(volatile ap_int<32> *fb1, AXI_STREAM &outs){
AP_AXIU32 dmad;
LOOP_Y1: for(int y=0; y<VGA_HEIGHT; y++){
LOOP_X1: for(int x=0; x<VGA_WIDTH; x++){
#pragma HLS PIPELINE II=1
dmad.data = fb1[(y*VGA_WIDTH)+x];
outs << dmad;
}
}
return(0);
}
int fb2_hls_stream(volatile ap_int<32> *fb2, AXI_STREAM &outs){
AP_AXIU32 dmad;
LOOP_Y1: for(int y=0; y<VGA_HEIGHT; y++){
LOOP_X1: for(int x=0; x<VGA_WIDTH; x++){
#pragma HLS PIPELINE II=1
dmad.data = fb2[(y*VGA_WIDTH)+x];
outs << dmad;
}
}
return(0);
}
// disp_dmar_axis_vga.cpp
// 2019/07/11 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height);
void pix_in_zero(AP_AXIU32 &pix, int x, int y, int max_width);
int disp_dmar_axis_vga(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<32> &resolution, ap_uint<2> &active_frame){
#pragma HLS INTERFACE ap_ctrl_hs port=return
#pragma HLS INTERFACE ap_none port=active_frame
#pragma HLS INTERFACE s_axilite port=resolution
#pragma HLS INTERFACE axis register both port=outs
#pragma HLS INTERFACE m_axi depth=307200 port=fb2 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb1 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb0 offset=slave
AP_AXIU32 pix;
int max_width, max_height;
switch((int)resolution){
case 0: // SVGA
max_width = SVGA_WIDTH;
max_height = SVGA_HEIGHT;
break;
case 1: // XGA
max_width = XGA_WIDTH;
max_height = XGA_HEIGHT;
break;
default: // HD
max_width = HD_WIDTH;
max_height = HD_HEIGHT;
break;
}
if (active_frame == (ap_uint<2>)0)
disp_dmar_fb2(fb2, outs, max_width, max_height);
else if (active_frame == (ap_uint<2>)1)
disp_dmar_fb0(fb0, outs, max_width, max_height);
else
disp_dmar_fb1(fb1, outs, max_width, max_height);
return(0);
}
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
Y00: for (int y=0; y<y_padding; y++){ // 最初の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X00: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y01: for (int y=y_padding; y<y_padding+VGA_HEIGHT; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
X010: for (int x=0; x<x_padding; x++){ // 最初の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
X011: for (int x=x_padding; x<x_padding+VGA_WIDTH; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb0[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
X012: for (int x=x_padding+VGA_WIDTH; x<max_width; x++){ // 最後の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y02: for (int y=y_padding+VGA_HEIGHT; y<max_height; y++){ // 最後の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X02: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
return(0);
}
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
Y10: for (int y=0; y<y_padding; y++){ // 最初の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X10: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y11: for (int y=y_padding; y<y_padding+VGA_HEIGHT; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
X100: for (int x=0; x<x_padding; x++){ // 最初の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
X101: for (int x=x_padding; x<x_padding+VGA_WIDTH; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb1[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
X102: for (int x=x_padding+VGA_WIDTH; x<max_width; x++){ // 最後の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y12: for (int y=y_padding+VGA_HEIGHT; y<max_height; y++){ // 最後の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X12: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
return(0);
}
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
Y20: for (int y=0; y<y_padding; y++){ // 最初の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X20: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y21: for (int y=y_padding; y<y_padding+VGA_HEIGHT; y++){
#pragma HLS LOOP_TRIPCOUNT min=480 max=480 avg=480
X210: for (int x=0; x<x_padding; x++){ // 最初の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
X211: for (int x=x_padding; x<x_padding+VGA_WIDTH; x++){
#pragma HLS LOOP_TRIPCOUNT min=640 max=640 avg=640
#pragma HLS PIPELINE II=1
pix.data = fb2[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
X212: for (int x=x_padding+VGA_WIDTH; x<max_width; x++){ // 最後の空白列
#pragma HLS LOOP_TRIPCOUNT min=80 max=640 avg=192
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
Y22: for (int y=y_padding+VGA_HEIGHT; y<max_height; y++){ // 最後の空白行
#pragma HLS LOOP_TRIPCOUNT min=60 max=300 avg=144
X22: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
pix_in_zero(pix, x, y, max_width);
outs << pix;
}
}
return(0);
}
void pix_in_zero(AP_AXIU32 &pix, int x, int y, int max_width){
pix.data = 0;
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
}
// disp_dmar_axis_vga.h
// 2019/07/11 by marsee
//
#ifndef __DISP_DMAR_AXIS_VGA_H__
#define __DISP_DMAR_AXIS_VGA_H__
#include "ap_axi_sdata.h"
#include "hls_video.h"
#define VGA_WIDTH 640
#define VGA_HEIGHT 480
#define SVGA_WIDTH 800
#define SVGA_HEIGHT 600
#define XGA_WIDTH 1024
#define XGA_HEIGHT 768
#define HD_WIDTH 1920
#define HD_HEIGHT 1080
#define RESO_SVGA 0
#define RESO_XGA 1
#define RESO_HD 2
typedef hls::stream<ap_axiu<32,1,1,1> > AXI_STREAM;
typedef ap_axiu<32,1,1,1> AP_AXIU32;
typedef hls::Scalar<3, unsigned char> RGB_PIXEL;
typedef hls::Mat<HD_HEIGHT, HD_WIDTH, HLS_8UC3> RGB_IMAGE;
typedef hls::Mat<HD_HEIGHT, HD_WIDTH, HLS_8UC1> GRAY_IMAGE;
#endif
// disp_dmar_axis_vga.cpp
// 2019/07/11 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <ap_axi_sdata.h>
#include "disp_dmar_axis_vga.h"
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height);
int disp_dmar_axis_vga(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<32> &resolution, ap_uint<2> &active_frame){
#pragma HLS INTERFACE ap_ctrl_hs port=return
#pragma HLS INTERFACE ap_none port=active_frame
#pragma HLS INTERFACE s_axilite port=resolution
#pragma HLS INTERFACE axis register both port=outs
#pragma HLS INTERFACE m_axi depth=307200 port=fb2 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb1 offset=slave
#pragma HLS INTERFACE m_axi depth=307200 port=fb0 offset=slave
AP_AXIU32 pix;
int max_width, max_height;
switch((int)resolution){
case 0: // SVGA
max_width = SVGA_WIDTH;
max_height = SVGA_HEIGHT;
break;
case 1: // XGA
max_width = XGA_WIDTH;
max_height = XGA_HEIGHT;
break;
default: // HD
max_width = HD_WIDTH;
max_height = HD_HEIGHT;
break;
}
if (active_frame == (ap_uint<2>)0)
disp_dmar_fb2(fb2, outs, max_width, max_height);
else if (active_frame == (ap_uint<2>)1)
disp_dmar_fb0(fb0, outs, max_width, max_height);
else
disp_dmar_fb1(fb1, outs, max_width, max_height);
return(0);
}
int disp_dmar_fb0(volatile ap_int<32> *fb0, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
LOOP_Y0: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=600 max=1080 avg=768
LOOP_X0: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
if (y < y_padding || y >= y_padding+VGA_HEIGHT)
pix.data = 0;
else if (x < x_padding || x >= x_padding+VGA_WIDTH)
pix.data = 0;
else
pix.data = fb0[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
int disp_dmar_fb1(volatile ap_int<32> *fb1, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
LOOP_Y1: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=600 max=1080 avg=768
LOOP_X1: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
if (y < y_padding || y >= y_padding+VGA_HEIGHT)
pix.data = 0;
else if (x < x_padding || x >= x_padding+VGA_WIDTH)
pix.data = 0;
else
pix.data = fb1[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
int disp_dmar_fb2(volatile ap_int<32> *fb2, AXI_STREAM &outs, int max_width, int max_height){
AP_AXIU32 pix;
int x_padding = (max_width - VGA_WIDTH)/2;
int y_padding = (max_height - VGA_HEIGHT)/2;
LOOP_Y2: for (int y=0; y<max_height; y++){
#pragma HLS LOOP_TRIPCOUNT min=600 max=1080 avg=768
LOOP_X2: for (int x=0; x<max_width; x++){
#pragma HLS LOOP_TRIPCOUNT min=800 max=1920 avg=1024
#pragma HLS PIPELINE II=1
if (y < y_padding || y >= y_padding+VGA_HEIGHT)
pix.data = 0;
else if (x < x_padding || x >= x_padding+VGA_WIDTH)
pix.data = 0;
else
pix.data = fb2[((y-y_padding)*VGA_WIDTH)+x-x_padding];
if (x==0 && y==0)
pix.user = 1;
else
pix.user = 0;
if (x == max_width-1)
pix.last = 1;
else
pix.last = 0;
outs << pix;
}
}
return(0);
}
// disp_dmar_axis_tb.cpp
// 2019/02/01 by marsee
//
#include <ap_int.h>
#include <hls_stream.h>
#include <iostream>
#include <fstream>
#include "hls_opencv.h"
#include "disp_dmar_axis_vga.h"
int disp_dmar_axis_vga(volatile ap_int<32> *fb0, volatile ap_int<32> *fb1, volatile ap_int<32> *fb2,
AXI_STREAM &outs, ap_uint<32> &resolution, ap_uint<2> &active_frame);
#define NUM_FRAME_BUFFER 3
#define FB_NUMBER 0
int main(){
using namespace cv;
ap_uint<2> fb_number = FB_NUMBER;
ap_uint<32> resolution = RESO_XGA;
AXI_STREAM outs;
AP_AXIU32 pix;
ap_uint<32> *frame_buffer;
// OpenCV で 画像を読み込む
Mat src = imread("bmp_file0.bmp");
// 画像をセーブするメモリをアロケート
if ((frame_buffer =(ap_uint<32> *)malloc(NUM_FRAME_BUFFER * sizeof(ap_int<32>) * (src.cols * src.rows))) == NULL){
fprintf(stderr, "Can't allocate frame_buffer0 ~ 2\n");
exit(1);
}
// Mat フォーマットからフレームバッファに画像をロード
Mat_<Vec3b> src_vec3b = Mat_<Vec3b>(src);
for(int i=0; i<NUM_FRAME_BUFFER; i++){
for(int y=0; y<src.rows; y++){
for(int x=0; x<src.cols; x++){
Vec3b pixel;
pixel = src_vec3b(y,x);
int rgb = ((int)(pixel(2))<<16) + ((int)(pixel(1))<<8) + pixel(0);
frame_buffer[(src.cols*src.rows)*i + (src.cols*y) + x] = rgb;
}
}
}
disp_dmar_axis_vga((volatile ap_int<32> *)frame_buffer,
(volatile ap_int<32> *)&frame_buffer[src.cols * src.rows],
(volatile ap_int<32> *)&frame_buffer[2 * (src.cols * src.rows)],
outs, resolution, fb_number);
// AXI4 Stream から Mat フォーマットに変換しファイルに書き込み
int row, col;
switch(resolution){
case RESO_SVGA:
row = SVGA_HEIGHT; col = SVGA_WIDTH;
break;
case RESO_XGA:
row = XGA_HEIGHT; col = XGA_WIDTH;
break;
default:
row = HD_HEIGHT; col = HD_WIDTH;
break;
}
Mat dst(row, col, CV_8UC3);
Mat_<Vec3b> dst_vec3b = Mat_<Vec3b>(dst);
for(int y=0; y<dst.rows; y++){
for(int x=0; x<dst.cols; x++){
outs >> pix;
int rgb = pix.data;
//int rgb = frame_buffer[(src.cols*y) + x];
Vec3b pixel;
pixel[0] = (rgb & 0xff); // blue
pixel[1] = (rgb & 0xff00) >> 8; // green
pixel[2] = (rgb & 0xff0000) >> 16; // red
dst_vec3b(y,x) = pixel;
}
}
imwrite("dmar_result.bmp", dst);
free(frame_buffer);
return(0);
}
all:
{
[destination_device = pl] design_1_wrapper.bit
}
KerasJson: example-keras-model-files/keras_curve_cnn2_line.json
KerasH5: example-keras-model-files/keras_curve_cnn2_line_weights.h5
OutputDir: keras_curve_cnn2_line
ProjectName: keras_curve_cnn2_line
XilinxPart: xcku5p-sfvb784-1-e
ClockPeriod: 5
IOType: io_parallel # options: io_serial/io_parallel
HLSConfig:
Model:
Precision: ap_fixed<16,6>
ReuseFactor: 1
ERROR: [XFORM 203-504] Stop unrolling loop 'Product1' (/home/masaaki/DNN/hls4ml/nnet_utils/nnet_dense.h:97) in function 'nnet::dense
, ap_fixed<16, 6, (ap_q_mode)5, (ap_o_mode)3, 0>, config6>' because it may cause large runtime and excessive memory usage due to increase in code size. Please avoid unrolling the loop or form sub-functions for code in the loop body.
ERROR: [HLS 200-70] Pre-synthesis failed.
command 'ap_source' returned error code
while executing
"source build_prj.tcl"
("uplevel" body line 1)
invoked from within
"uplevel \#0 [list source $arg] "
# 学習済みモデルの読み込み
from keras.models import load_model
model = load_model('keras_curve_cnn2_line.h5')
# save as JSON
json_string = model.to_json()
json_name='keras_curve_cnn2_line.json'
with open(json_name, mode='w') as f:
f.write(json_string)
model.save_weights('keras_curve_cnn2_line_weights.h5')
//
// rfnoc-hls-neuralnet: Vivado HLS code for neural-net building blocks
//
// Copyright (C) 2017 EJ Kreinar
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
#include <iostream>
#include "KERAS_conv1d_small_nfilt5.h"
//hls-fpga-machine-learning insert weights
#include "weights/w2.h"
#include "weights/b2.h"
#include "weights/w4.h"
#include "weights/b4.h"
#include "weights/w6.h"
#include "weights/b6.h"
#include "weights/w8.h"
#include "weights/b8.h"
#include "weights/w10.h"
#include "weights/b10.h"
void KERAS_conv1d_small_nfilt5(
input_t input_1[N_INPUT_1_1*N_INPUT_2_1],
result_t layer11_out[N_LAYER_10],
unsigned short &const_size_in_1,
unsigned short &const_size_out_1
) {
//hls-fpga-machine-learning insert IO
#pragma HLS ARRAY_RESHAPE variable=input_1 complete dim=0
#pragma HLS ARRAY_RESHAPE variable=layer11_out complete dim=0
#pragma HLS INTERFACE ap_vld port=input_1,layer11_out
#pragma HLS PIPELINE
const_size_in_1 = N_INPUT_1_1*N_INPUT_2_1;
const_size_out_1 = N_LAYER_10;
// ****************************************
// NETWORK INSTANTIATION
// ****************************************
//hls-fpga-machine-learning insert layers
layer2_t layer2_out[Y_OUTPUTS_2*N_FILT_2];
#pragma HLS ARRAY_PARTITION variable=layer2_out complete dim=0
nnet::conv_1d<input_t, layer2_t, config2>(input_1, layer2_out, w2, b2);
layer3_t layer3_out[Y_OUTPUTS_2*N_FILT_2];
#pragma HLS ARRAY_PARTITION variable=layer3_out complete dim=0
nnet::relu<layer2_t, layer3_t, relu_config3>(layer2_out, layer3_out);
layer4_t layer4_out[Y_OUTPUTS_4*N_FILT_4];
#pragma HLS ARRAY_PARTITION variable=layer4_out complete dim=0
nnet::conv_1d<layer3_t, layer4_t, config4>(layer3_out, layer4_out, w4, b4);
layer5_t layer5_out[Y_OUTPUTS_4*N_FILT_4];
#pragma HLS ARRAY_PARTITION variable=layer5_out complete dim=0
nnet::relu<layer4_t, layer5_t, relu_config5>(layer4_out, layer5_out);
layer6_t layer6_out[Y_OUTPUTS_6*N_FILT_6];
#pragma HLS ARRAY_PARTITION variable=layer6_out complete dim=0
nnet::conv_1d<layer5_t, layer6_t, config6>(layer5_out, layer6_out, w6, b6);
layer7_t layer7_out[Y_OUTPUTS_6*N_FILT_6];
#pragma HLS ARRAY_PARTITION variable=layer7_out complete dim=0
nnet::relu<layer6_t, layer7_t, relu_config7>(layer6_out, layer7_out);
layer8_t layer8_out[N_LAYER_8];
#pragma HLS ARRAY_PARTITION variable=layer8_out complete dim=0
nnet::dense<layer7_t, layer8_t, config8>(layer7_out, layer8_out, w8, b8);
layer9_t layer9_out[N_LAYER_8];
#pragma HLS ARRAY_PARTITION variable=layer9_out complete dim=0
nnet::relu<layer8_t, layer9_t, relu_config9>(layer8_out, layer9_out);
layer10_t layer10_out[N_LAYER_10];
#pragma HLS ARRAY_PARTITION variable=layer10_out complete dim=0
nnet::dense<layer9_t, layer10_t, config10>(layer9_out, layer10_out, w10, b10);
nnet::softmax<layer10_t, result_t, softmax_config11>(layer10_out, layer11_out);
}
入力層 40 入力
conv_1d 入力数 40 出力数 30
relu 入力数 30 出力数 30
conv_1d 入力数 30 出力数 20
relu 入力数 20 出力数 20
conv_1d 入力数 20 出力数 10
relu 入力数 10 出力数 10
dense 入力数 10 出力数 5
relu 入力数 5 出力数 5
dense 入力数 5 出力数 5
softmax 入力数 5 出力数 5
entity KERAS_conv1d_small_nfilt5 is
port (
ap_clk : IN STD_LOGIC;
ap_rst : IN STD_LOGIC;
ap_start : IN STD_LOGIC;
ap_done : OUT STD_LOGIC;
ap_idle : OUT STD_LOGIC;
ap_ready : OUT STD_LOGIC;
input_1_V_ap_vld : IN STD_LOGIC;
input_1_V : IN STD_LOGIC_VECTOR (639 downto 0);
layer11_out_V : OUT STD_LOGIC_VECTOR (79 downto 0);
layer11_out_V_ap_vld : OUT STD_LOGIC;
const_size_in_1 : OUT STD_LOGIC_VECTOR (15 downto 0);
const_size_in_1_ap_vld : OUT STD_LOGIC;
const_size_out_1 : OUT STD_LOGIC_VECTOR (15 downto 0);
const_size_out_1_ap_vld : OUT STD_LOGIC );
end;
KerasJson: example-keras-model-files/KERAS_conv1d_small_nfilt5.json
KerasH5: example-keras-model-files/KERAS_conv1d_small_nfilt5_weights.h5
OutputDir: KERAS_conv1d_small_nfilt5
ProjectName: KERAS_conv1d_small_nfilt5
XilinxPart: xcku5p-sfvb784-1-e
ClockPeriod: 5
IOType: io_parallel # options: io_serial/io_parallel
HLSConfig:
Model:
Precision: ap_fixed<16,6>
ReuseFactor: 1
//
// rfnoc-hls-neuralnet: Vivado HLS code for neural-net building blocks
//
// Copyright (C) 2017 EJ Kreinar
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
#include <fstream>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "firmware/parameters.h"
#include "firmware/myproject.h"
#include "nnet_helpers.h"
int main(int argc, char **argv)
{
//hls-fpga-machine-learning insert data
input_t input_1[N_INPUT_1_1] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
result_t layer9_out[N_LAYER_8] = {0,0,0,0,0};
//hls-fpga-machine-learning insert top-level-function
unsigned short size_in1,size_out1;
myproject(input_1,layer9_out,size_in1,size_out1);
//hls-fpga-machine-learning insert output
for(int i = 0; i < N_LAYER_8; i++) {
std::cout << layer9_out[i] << " ";
}
std::cout << std::endl;
return 0;
}
INFO: [SIM 2] *************** CSIM start ***************
INFO: [SIM 4] CSIM will launch GCC as the compiler.
Compiling ../../../../myproject_test.cpp in debug mode
Compiling ../../../../firmware/myproject.cpp in debug mode
Generating csim.exe
0.0292969 0.761719 0.0566406 0.142578 0.0361328
INFO: [SIM 1] CSim done with 0 errors.
INFO: [SIM 3] *************** CSIM finish ***************
//
// rfnoc-hls-neuralnet: Vivado HLS code for neural-net building blocks
//
// Copyright (C) 2017 EJ Kreinar
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
#include <fstream>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "firmware/parameters.h"
#include "firmware/myproject.h"
#include "nnet_helpers.h"
int main(int argc, char **argv)
{
//hls-fpga-machine-learning insert data
input_t input_1[N_INPUT_1_1] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
input_t input_2[N_INPUT_1_1] = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16};
result_t layer9_out[N_LAYER_8] = {0,0,0,0,0};
result_t layer9_out2[N_LAYER_8] = {0,0,0,0,0};
//hls-fpga-machine-learning insert top-level-function
unsigned short size_in1,size_out1;
myproject(input_1,layer9_out,size_in1,size_out1);
myproject(input_2,layer9_out2,size_in1,size_out1);
//hls-fpga-machine-learning insert output
for(int i = 0; i < N_LAYER_8; i++) {
std::cout << layer9_out[i] << " ";
}
std::cout << std::endl;
for(int i = 0; i < N_LAYER_8; i++) {
std::cout << layer9_out2[i] << " ";
}
std::cout << std::endl;
return 0;
}
INFO: [SIM 2] *************** CSIM start ***************
INFO: [SIM 4] CSIM will launch GCC as the compiler.
Compiling ../../../../myproject_test.cpp in debug mode
Generating csim.exe
0.0292969 0.761719 0.0566406 0.142578 0.0361328
0 0.399414 0.0439453 0.571289 0.015625
INFO: [SIM 1] CSim done with 0 errors.
INFO: [SIM 3] *************** CSIM finish ***************
日 | 月 | 火 | 水 | 木 | 金 | 土 |
---|---|---|---|---|---|---|
- | 1 | 2 | 3 | 4 | 5 | 6 |
7 | 8 | 9 | 10 | 11 | 12 | 13 |
14 | 15 | 16 | 17 | 18 | 19 | 20 |
21 | 22 | 23 | 24 | 25 | 26 | 27 |
28 | 29 | 30 | 31 | - | - | - |