机器找不到 libcudnn.so.6

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 72, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

.bashrc 中加入

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

pix2code GUI转换为 iOS,android,web 页面源码

GITHUB:  https://github.com/tonybeltramelli/pix2code

操作流程见github,非常详细。

一路无话,在公司用 1070训,报

python3 ./train.py ../datasets/web/training_set ../bin
Using TensorFlow backend.
Loading data...
Generating sparse vectors...
Dataset size: 143741
Vocabulary size: 19
Input shape: (256, 256, 3)
Output size: 19
Convert arrays...
Traceback (most recent call last):
 File "./train.py", line 66, in <module>
 run(input_path, output_path, is_memory_intensive=use_generator, pretrained_model=pretrained_weigths)
 File "./train.py", line 24, in run
 dataset.convert_arrays()
 File "/root/code/pix2code/model/classes/dataset/Dataset.py", line 82, in convert_arrays
 self.input_images = np.array(self.input_images)
MemoryError

nvidia-smi 了下,发现有个进程占了2G显存。kill 之,重试还报一样的错误。

重新看了下 github 上的 readme

解决方案,用下面这个语句。也是官方推荐的train方式,之前没仔细看。

python3 ./train.py ../datasets/web/training_set ../bin 1

开始非常愉快的训练了。。。1080ti 大概 18分钟1轮,默认设定为 10轮。

一夜无话。。。。到早上终于训练好了

截了一张博客的图

生成 GUI

python3 ./sample.py ../bin pix2code  1.jpg  ../code

生成的GUI,似乎生成的是它演示的哪个页面。不是我提供的图。。奇怪。重新训一下试试。

header{
btn-inactive,btn-active
}
row{
single{
small-title,text,btn-green
}
}
row{
quadruple{
small-title,text,btn-orange
}
quadruple{
small-title,text,btn-orange
}
quadruple{
small-title,text,btn-orange
}
quadruple{
small-title,text,btn-orange
}
}
row{
double{
small-title,text,btn-orange
}
double{
small-title,text,btn-orange
}
}

忘了加  python3 ,默认用 python2 训完了报错

Traceback (most recent call last):
  File "./train.py", line 66, in <module>
    run(input_path, output_path, is_memory_intensive=use_generator, pretrained_m
odel=pretrained_weigths)
  File "./train.py", line 51, in run
    model.fit_generator(generator, steps_per_epoch=steps_per_epoch)
  File "/home/endler/code/pix2code/model/classes/model/pix2code.py", line 70, in
 fit_generator
    self.save()
  File "/home/endler/code/pix2code/model/classes/model/AModel.py", line 18, in s
ave
    self.model.save_weights("{}/{}.h5".format(self.output_path, self.name))
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2
580, in save_weights
    raise ImportError('`save_weights` requires h5py.')
ImportError: `save_weights` requires h5py.

哭死,python2.7 没装  h5py。。。还得重训一次。。。。

gogogo,我就不信了。。。

又报这个,不过 权重文件已经生成了。

Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7f3f6f67ae80>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 696, in __del__
TypeError: 'NoneType' object is not callable

反复试验了几次,貌似它只能识别 demo 哪几张图。ㄟ( ▔, ▔ )ㄏ

Age Gender Estimate TF 一个tensorflow 识别年龄的demo

GITHUB:https://github.com//BoyuanJiang/Age-Gender-Estimate-TF

clone 代码。

下载数据集 到  ~/data

wget   https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/static/imdb_crop.tar

加压缩

tar -xvf imdb_crop.tar

数据集预处理

python convert_to_records_multiCPU.py --imdb --nworks 8

训练的时候发现输出的文件为空。

try 了一下

example = tf.train.Example(features=tf.train.Features(feature={
 # 'height': _int64_feature(rows),
 # 'width': _int64_feature(cols),
 # 'depth': _int64_feature(depth),
 'age': _int64_feature(int(ages[index])),
 'gender': _int64_feature(int(genders[index])),
 'image_raw': _bytes_feature(image_raw),
 'file_name': _bytes_feature(str(file_name[index][0]))}))

报错

’40/nm1102140_rm3713850624_1974-7-29_2013.jpg’ has type <class ‘str’>, but expected one of: ((<class ‘bytes’>,),)

去掉

'file_name': _bytes_feature(str(file_name[index][0]))

程序可以正常跑了
训练的时候报错

Invalid argument: Name: <unknown>, Feature: file_name (data type: st
ring) is required but could not be found.

修改 file_name 行如下

'file_name': _bytes_feature(bytes(file_name[index][0],'utf-8'))}))

重新生成。。。

或者换成

python convert_to_records.py –imdb

就好了。

下载models:https://pan.baidu.com/s/1dFewgqH

训练好的model

https://pan.baidu.com/s/1bpllJg7

Train

python3  train.py --lr 1e-3 --weight_decay 1e-5 --epoch 6 --batch_size 128 --keep_prob 0.8 --cuda

Test 选出最好的模型

python3 test.py --images "./data/test" --model_path "./models" --batch_size 128 --choose_best --cuda
Age_MAE:7.07,Gender_Acc:80.92%,Age_model:./models/model.ckpt-12001,Gender_model:./models/model.ckpt-9001

用自己的图片测试

python3 eval.py --I "./demo/demo.jpg" --M "./models/" --font_scale 1 --thickness 1

 

KittiSeg Road Segmentation model implemented in tensorflow.

GITHUB:https://github.com/MarvinTeichmann/KittiSeg

git clone https://github.com/MarvinTeichmann/KittiSeg.gi
cd KittiSeg
git submodule update --init --recursive

python download_data.py --kitti_url  http://kitti.is.tue.mpg.de/kitti/data_road.zip

python3 demo.py --input_image data/demo/demo.png

自己找了几张图测试,效果还行。

 

待解决:

  1. 对非公路如石子路识别不理想。
  2. 公路车多的时候,检测不出公路
  3. 识别的时候,似乎没有使用GPU。
    1. 原因是我机器上装了 tensorflow cpu 和gpu 版,gpu 是 1.2  cpu 是1.4 这个项目在1.2上跑不起来。都卸了重装最新版的 tensorflow-gpu。
    2. 安装的时候报错。
    3.  libcudnn.so.6: cannot open shared object file: No such file or directory
    4. https://developer.nvidia.com/rdp/cudnn-download 下载新的 cudnn 安装。
    5. Navigate to your <cudnnpath> directory containing the cuDNN Tar file.
    6. Unzip the cuDNN package.
      $ tar -xzvf cudnn-9.0-linux-x64-v7.tgz
    7. Copy the following files into the CUDA Toolkit directory.
      $ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
      $ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
      $ sudo chmod a+r /usr/local/cuda/include/cudnn.h
      /usr/local/cuda/lib64/libcudnn*
      
      
    8.  sudo ldconfig /usr/local/cuda/lib64  #没有这步,还会出错。
    9. 2017-12-15 12:25:14.185527: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
      
      name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
      
      pciBusID: 0000:01:00.0
      
      totalMemory: 10.90GiB freeMemory: 10.46GiB
      
      2017-12-15 12:25:14.185550: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
    10. 问题解决了

Self-Driving-Car-Demo 第一个无人驾驶Demo

5分钟视频系列,一个最简单的无人驾驶 demo。

GITHUB:https://github.com/llSourcell/Self-Driving-Car-Demo

环境准备

sudo apt-get install mercurial
hg clone https://bitbucket.org/pygame/pygame
cd pygame
sudo apt-get install python3-dev python3-numpy libsdl-dev libsdl-image1.2-dev \
  libsdl-mixer1.2-dev libsdl-ttf2.0-dev libsmpeg-dev libportmidi-dev \
  libavformat-dev libswscale-dev libjpeg-dev libfreetype6-dev
python3 setup.py build
sudo python3 setup.py install
wget https://github.com/viblo/pymunk/archive/pymunk-4.0.0.zip
pip install pymunk-4.0.0.zip
pip3 install keras
pip3 install git+git://github.com/Theano/Theano.git --upgrade --no-deps
pip3 install h5py

到  pymunk 目录  将 python2 的代码格式转为 python3 的

cd /usr/local/lib/python3.5/dist-packages/pymunk

2to3 -w *.py

回到  Self-Driving-Car-Demo 目录

python3 learning.py
报错
pygame.error: No video mode large enough for 1000x700

修改文件 /flat_game/carmunk.py  根据显卡情况修改。1070 的破卡,所以改的小了点。

width = 500 #1000
height = 350 #700

漫长的训练ing   ( 根据说明大概需要 36 hour)

25000 轮后报错 /(ㄒoㄒ)/~~

OSError: Unable to create file (unable to open file: name = ‘saved-models/164-150-100-50000-25000.h5’, errn
o = 2, error message = ‘No such file or directory’, flags = 13, o_flags = 242)

创建  saved-models 文件夹。重新训练。。。。

pip3 install matplotlib
apt install python3-tk
python3 plotting.py

修改  nn.py 中 model 的名字。

python3 playing.py

 

Tensorflow Android Demo 使用 Yolo

它默认支持三种网络,

TF_OD_API, MULTIBOX, YOLO;

除了 YOLO ,都给了下载链接,想试试 YOLO ,没找到

graph-tiny-yolo-voc.pb

的下载,自己动手丰衣足食。

git clone git@github.com:thtrieu/darkflow.git
cd darkflow
python3 setup.py build_ext --inplace
pip install -e .

下载对应的 yolo weight 文件

wget https://pjreddie.com/media/files/yolo.weights

更多权重文件下载 :https://drive.google.com/drive/folders/0B1tW_VtY7onidEwyQ2FtQVplWEU

https://pjreddie.com/darknet/yolo/

flow --model cfg/yolo.cfg --load yolo.weights --savepb
Parsing ./cfg/yolo.cfg
Parsing cfg/yolo.cfg
Loading yolo.weights ...
Successfully identified 203934260 bytes
Finished in 0.008934736251831055s
Model has a coco model name, loading coco labels.

Building net ...
Source | Train? | Layer description | Output size
-------+--------+----------------------------------+---------------
 | | input | (?, 608, 608, 3)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 608, 608, 32)
 Load | Yep! | maxp 2x2p0_2 | (?, 304, 304, 32)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 304, 304, 64)
 Load | Yep! | maxp 2x2p0_2 | (?, 152, 152, 64)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 152, 152, 128)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 152, 152, 64)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 152, 152, 128)
 Load | Yep! | maxp 2x2p0_2 | (?, 76, 76, 128)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 76, 76, 256)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 76, 76, 128)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 76, 76, 256)
 Load | Yep! | maxp 2x2p0_2 | (?, 38, 38, 256)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 38, 38, 512)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 38, 38, 256)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 38, 38, 512)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 38, 38, 256)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 38, 38, 512)
 Load | Yep! | maxp 2x2p0_2 | (?, 19, 19, 512)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 19, 19, 512)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 19, 19, 512)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | concat [16] | (?, 38, 38, 512)
 Load | Yep! | conv 1x1p0_1 +bnorm leaky | (?, 38, 38, 64)
 Load | Yep! | local flatten 2x2 | (?, 19, 19, 256)
 Load | Yep! | concat [27, 24] | (?, 19, 19, 1280)
 Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 19, 19, 1024)
 Load | Yep! | conv 1x1p0_1 linear | (?, 19, 19, 425)
-------+--------+----------------------------------+---------------
Running entirely on CPU
2017-12-08 17:49:27.012124: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Finished in 5.091068267822266s

Rebuild a constant version ...
Done

训练好的文件在 ./built_graph/ 文件夹

将生成的文件拷贝到

android_asset

编译成功。。。

效果不是很理想,很多东西被认成  person 了。。。

TensorFlow Lite iOS 调试笔记

下载 tensorflow 代码,到目录

/tensorflow/contrib/lite/examples/ios/simple/

 

安装pods

sudo gem install cocoapods
pod install --verbose --no-repo-update

漫长的等待后打开

simple.xcworkspace

报错

tensorflow/contrib/lite/schema/schema_generated.h:7:10: 'flatbuffers/flatbuffers.h' file not found

FlatBuffers 没装

xcode-select --install
brew install automake
brew install libtool

下面命令需要退到 根目录下执行,否则找不到对应目录,另需已安装 wget

tensorflow/contrib/lite/download_dependencies.sh

在 cp 前 莫名 有个 echo ,导致 cp 不执行。需要手动下载这些包,再传到制定目录。

tensorflow/contrib/lite/build_ios_universal_lib.sh

编译不成功,感觉 这个 lite 版的 ios 还没有完工。也可能是我这台 imac的问题,回家用macpro 试试再说。

TensorFlow Demo labels 文件中文版(汉化完成)

使用相关: TensorFlow Lite 初探

试了下各种自动翻译,效果都不理想。还是自己动手丰衣足食。顺带背单词了。

手工翻了一半,用有道翻译了一半。2017.12.7 发现错误会随时更新。

12.8 更新了3个有道翻错的词。
继续阅读TensorFlow Demo labels 文件中文版(汉化完成)

mozilla语音识别框架 deepspeech 初探

安装

pip3 install deepspeech-gpu

运行

deepspeech output_model.pb my_audio_file.wav alphabet.txt

deepspeech output_graph.pb LDC93S1.wav alphabet.txt

报错

2017-12-01 09:35:28.879923: F tensorflow/core/platform/cpu_feature_guard.cc:35] The TensorFlow library was compiled to use AVX2 instructions, but these aren't available on your machine.
Aborted (core dumped)

issue 说明 :

https://github.com/mozilla/DeepSpeech/issues/1023

lscpu

Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz

CPU 只有 avx 指令集,没有 avx2

换到 阿里云的机器,CPU版本是 E5-2682 .可以用

pip3 install deepspeech

deepspeech output_graph.pb LDC93S1.wav alphabet.txt

output_graph.pb 下载地址

wget  https://s3.amazonaws.com/deep-speech/output_graph.pb

LDC93S1.wav  在

https://github.com/mozilla/DeepSpeech/tree/master/data/smoke_test

自己录了几段都不认,对音频格式了解不多,留个坑以后填。

alphabet.txt 在

wget https://raw.githubusercontent.com/mozilla/DeepSpeech/master/data/alphabet.txt

识别的结果:

(CPU,5.394s) shehadyeducksoingrecywachworallyear

(GPU,1.286s) she had ye duck so ingrecy wachwor all year

不知道为毛线,CPU版没空格,GPU版有。

原文对照:

She had your dark suit in greasy wash water all year.