Colab配置版

时间：2024年3月22日-2024年4月1日


# 最新代码复制运行点
wget https://file.cz123.top/3PhD/4BDA/Codes/ToBOML_Colab.sh && bash ToBOML_Colab.sh


x
#/bin/bash

apt-get -y update
apt-get -y upgrade

echo "更新完成"
# 需要安装ssl
apt -y install libssl-dev
apt -y install libffi-dev

echo "开始下载Python3.7"
wget https://www.python.org/ftp/python/3.7.11/Python-3.7.11.tgz
tar -zxf Python-3.7.11.tgz

echo "安装Python3.7.11"
cd ./Python-3.7.11
./configure
make && make install

cd ~
rm Python-3.7.11.tgz

echo "创建新环境"
cd ~
python3 -m venv tf-env-BDA
source ~/tf-env-BDA/bin/activate

# 升级git到最新
pip install --upgrade pip

# 安装tensorflow-1.14版本
cd ~

wget https://files.pythonhosted.org/packages/f4/28/96efba1a516cdacc2e2d6d081f699c001d414cc8ca3250e6d59ae657eb2b/tensorflow-1.14.0-cp37-cp37m-manylinux1_x86_64.whl
pip install tensorflow-1.14.0-cp37-cp37m-manylinux1_x86_64.whl
wget https://files.pythonhosted.org/packages/32/67/559ca8408431c37ad3a17e859c8c291ea82f092354074baef482b98ffb7b/tensorflow_gpu-1.14.0-cp37-cp37m-manylinux1_x86_64.whl
pip install tensorflow_gpu-1.14.0-cp37-cp37m-manylinux1_x86_64.whl

pip install protobuf==3.20.*

# pip install tensorflow==1.14
# pip install tensorflow-gpu==1.14

# 下载BOML代码
cd
git clone https://github.com/dut-media-lab/BOML.git

# 安装依赖包
# 其中要求tensorflow的版本为1.13.*到1.15.*
cd ~/BOML
pip install -r requirements.txt

检测gpu的方法


xxxxxxxxxx
# 查看tf版本信息
python -c "import tensorflow as tf; print(tf.__version__)"

# 查看Cuda版本
nvcc -V

# 查看gpu信息
nvidia-smi

# 查看系统版本等
uname -m && cat /etc/*release

Colab配置问题

BUG参考链接


xxxxxxxxxx

export LD_LIBRARY_PATH=/usr/localcuda/lib64

cuda安装问题

https://zhuanlan.zhihu.com/p/487941231

gcc查看和安装

gcc的安装：https://blog.csdn.net/qq_17783559/article/details/132385887


xxxxxxxxxx
#查看显卡驱动
nvidia-smi

#安装gcc-7
apt-get install gcc-7 g++-7

#查看gcc版本
ls /usr/bin/gcc*

# 更新apt
echo "deb [arch=amd64] http://archive.ubuntu.com/ubuntu focal main universe" >> /etc/apt/sources.list

# 安装
apt update
apt-get install gcc-7 g++-7

# 切换版本
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 80
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 80

方法1


xxxxxxxxxx
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1810/x86_64/cuda-repo-ubuntu1810_10.1.105-1_amd64.deb
dpkg -i cuda-repo-ubuntu1810_10.1.105-1_amd64.deb
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1810/x86_64/7fa2af80.pub
apt-get update
apt-get install cuda

方法2


xxxxxxxxxx
wget https://developer.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.105_418.39_linux.run
bash cuda_10.1.105_418.39_linux.run

方法3


xxxxxxxxxx
wget https://developer.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-ubuntu1810-10-1-local-10.1.105-418.39_1.0-1_amd64.deb
dpkg -i cuda-repo-ubuntu1810-10-1-local-10.1.105-418.39_1.0-1_amd64.deb
apt-key add /var/cuda-repo-<version>/7fa2af80.pub
apt-get update
apt-get install cuda

方法3的错误


xxxxxxxxxx
Errors were encountered while processing:
 nvidia-dkms-550
 nvidia-driver-550
 cuda-drivers-550
 cuda-drivers
 cuda-runtime-12-4
 cuda-12-4
 cuda
E: Sub-process /usr/bin/dpkg returned an error code (1)

方法2错误见文件


xxxxxxxxxx
# 测试gpu的可用性

import tensorflow as tf 

# tf.ConfigProto()主要的作用是配置tf.Session的运算方式
config = tf.ConfigProto()
config.gpu_options.allow_growth = True  # 不全部占满显存, 按需分配
config.gpu_options.per_process_gpu_memory_fraction = 0.6  # 限制GPU内存占用率
sess = tf.Session(config=config)

运行上述代码出现的bug提示见文件

安装前先卸载原来的cuda 12.1


xxxxxxxxxx
apt remove cuda

安装cuda 10.0

参考地址：https://blog.csdn.net/weixin_45629790/article/details/114803545

安装代码1(deb,network)：


xxxxxxxxxx
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
apt-get update
apt-get install cuda

安装代码2(runfile, local,2G)：


xxxxxxxxxx
wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linux
mv cuda_10.0.130_410.48_linux cuda_10.0.130_410.48_linux.run
sh cuda_10.0.130_410.48_linux.run

安装代码3(deb, local, 1.6G):


xxxxxxxxxx
wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64
dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb
apt-key add /var/cuda-repo-<version>/7fa2af80.pub
apt-get update
apt-get install cuda

安装cuDNN

参考地址：https://blog.csdn.net/m0_37605642/article/details/119637836

https://developer.nvidia.com/rdp/cudnn-archive


xxxxxxxxxx
wget http://104.244.90.25:12345/libcudnn7_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7_7.6.5.32-1+cuda10.0_amd64.deb

wget http://104.244.90.25:12345/libcudnn7-dev_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7-dev_7.6.5.32-1+cuda10.0_amd64.deb

wget http://104.244.90.25:12345/libcudnn7-doc_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7-doc_7.6.5.32-1+cuda10.0_amd64.deb

整个流程


xxxxxxxxxx
# 安装gcc

# 更新apt
echo "deb [arch=amd64] http://archive.ubuntu.com/ubuntu focal main universe" >> /etc/apt/sources.list

# 安装
apt update
apt-get install gcc-7 g++-7

# 切换版本
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 80
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 80


# 卸载 cuda 12.2
rm -rf /usr/local/cuda-12.2
rm /usr/local/cuda
rm /usr/local/cuda-12
apt autoremove 

# 安装 cuda 10.0 (方法2)
# 该方法需要手动选择
cd
wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linux
mv cuda_10.0.130_410.48_linux cuda_10.0.130_410.48_linux.run
sh cuda_10.0.130_410.48_linux.run
rm cuda_10.0.130_410.48_linux.run

# cuda的环境配置
echo "export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> ~/.bashrc
source ~/.bashrc

# 安装 cuDNN
wget http://104.244.90.25:12345/libcudnn7_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7_7.6.5.32-1+cuda10.0_amd64.deb

wget http://104.244.90.25:12345/libcudnn7-dev_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7-dev_7.6.5.32-1+cuda10.0_amd64.deb

wget http://104.244.90.25:12345/libcudnn7-doc_7.6.5.32-1+cuda10.0_amd64.deb
dpkg -i libcudnn7-doc_7.6.5.32-1+cuda10.0_amd64.deb