Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bonito basecaller missing --use_openvino flag #229

Closed
shishir-reddy opened this issue Feb 2, 2022 · 10 comments
Closed

Bonito basecaller missing --use_openvino flag #229

shishir-reddy opened this issue Feb 2, 2022 · 10 comments
Assignees
@iiSeymour

Comments

@shishir-reddy
Copy link

shishir-reddy commented Feb 2, 2022 •
edited
Loading

The pip install of ont-bonito completes successfully for me, but when I try using the cpu version with the flag --use_openvino I see an error.

(bvenv) root@t1v-n-fead2fe2-w-0:/home/test# pip install --upgrade pip
Collecting pip
  Using cached pip-22.0.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-22.0.2
(bvenv) root@t1v-n-fead2fe2-w-0:/home/test# pip install ont-bonito
Requirement already satisfied: ont-bonito in ./bvenv/lib/python3.8/site-packages (0.5.0)
Requirement already satisfied: cupy-cuda102==9.6.0 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (9.6.0)
Requirement already satisfied: fast-ctc-decode==0.3.0 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.3.0)
Requirement already satisfied: koi-cuda102==0.0.5 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.0.5)
Requirement already satisfied: mappy==2.23 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (2.23)
Requirement already satisfied: numpy~=1.19.5 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (1.19.5)
Requirement already satisfied: ont-fast5-api==3.3.0 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (3.3.0)
Requirement already satisfied: ont-remora==0.1.1 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.1.1)
Requirement already satisfied: ont-seqdist-cuda102==0.0.4 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.0.4)
Requirement already satisfied: pandas==1.1.5 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (1.1.5)
Requirement already satisfied: parasail==1.2 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (1.2)
Requirement already satisfied: pysam==0.18.0 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.18.0)
Requirement already satisfied: requests==2.25.1 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (2.25.1)
Requirement already satisfied: scipy==1.5.4 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (1.5.4)
Requirement already satisfied: toml==0.10.2 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (0.10.2)
Requirement already satisfied: torch~=1.10.0 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (1.10.2)
Requirement already satisfied: tqdm==4.31.1 in ./bvenv/lib/python3.8/site-packages (from ont-bonito) (4.31.1)
Requirement already satisfied: fastrlock>=0.5 in ./bvenv/lib/python3.8/site-packages (from cupy-cuda102==9.6.0->ont-bonito) (0.8)
Requirement already satisfied: cffi in ./bvenv/lib/python3.8/site-packages (from koi-cuda102==0.0.5->ont-bonito) (1.15.0)
Requirement already satisfied: h5py>=2.8 in ./bvenv/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (3.6.0)
Requirement already satisfied: progressbar33>=2.3.1 in ./bvenv/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (2.4)
Requirement already satisfied: packaging in ./bvenv/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (21.3)
Requirement already satisfied: tabulate in ./bvenv/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito) (0.8.9)
Requirement already satisfied: thop in ./bvenv/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito) (0.0.31.post2005241907)
Requirement already satisfied: onnx in ./bvenv/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito) (1.10.2)
Requirement already satisfied: scikit-learn in ./bvenv/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito) (1.0.2)
Requirement already satisfied: onnxruntime>=1.7 in ./bvenv/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito) (1.10.0)
Requirement already satisfied: python-dateutil>=2.7.3 in ./bvenv/lib/python3.8/site-packages (from pandas==1.1.5->ont-bonito) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in ./bvenv/lib/python3.8/site-packages (from pandas==1.1.5->ont-bonito) (2021.3)
Requirement already satisfied: wheel in ./bvenv/lib/python3.8/site-packages (from parasail==1.2->ont-bonito) (0.37.1)
Requirement already satisfied: certifi>=2017.4.17 in ./bvenv/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./bvenv/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (1.26.8)
Requirement already satisfied: chardet<5,>=3.0.2 in ./bvenv/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in ./bvenv/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2.10)
Requirement already satisfied: typing-extensions in ./bvenv/lib/python3.8/site-packages (from torch~=1.10.0->ont-bonito) (4.0.1)
Requirement already satisfied: flatbuffers in ./bvenv/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito) (2.0)
Requirement already satisfied: protobuf in ./bvenv/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito) (3.19.4)
Requirement already satisfied: six>=1.5 in ./bvenv/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas==1.1.5->ont-bonito) (1.16.0)
Requirement already satisfied: pycparser in ./bvenv/lib/python3.8/site-packages (from cffi->koi-cuda102==0.0.5->ont-bonito) (2.21)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./bvenv/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito) (3.0.7)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./bvenv/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito) (3.1.0)
Requirement already satisfied: joblib>=0.11 in ./bvenv/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito) (1.1.0)

But when I try running bonito I get the following error:

(bvenv) root@t1v-n-fead2fe2-w-0:/home/test# bonito basecaller [email protected] --use_openvino --device=cpu ./fast5/ > basecall.fastq
usage: bonito [-h] [-v]
              {basecaller,train,evaluate,view,convert,download,export} ...
bonito: error: unrecognized arguments: --use_openvino
(bvenv) root@t1v-n-fead2fe2-w-0:/home/test# bonito basecaller -h
usage: bonito basecaller [-h] [--reference REFERENCE] [--modified-bases MODIFIED_BASES [MODIFIED_BASES ...]] [--modified-base-model MODIFIED_BASE_MODEL]
                         [--read-ids READ_IDS] [--device DEVICE] [--seed SEED] [--weights WEIGHTS] [--skip] [--save-ctc] [--revcomp] [--recursive]
                         [--quantize | --no-quantize] [--overlap OVERLAP] [--chunksize CHUNKSIZE] [--batchsize BATCHSIZE] [--max-reads MAX_READS] [-v]
                         model_directory reads_directory

positional arguments:
  model_directory
  reads_directory

optional arguments:
  -h, --help            show this help message and exit
  --reference REFERENCE
  --modified-bases MODIFIED_BASES [MODIFIED_BASES ...]
  --modified-base-model MODIFIED_BASE_MODEL
  --read-ids READ_IDS
  --device DEVICE
  --seed SEED
  --weights WEIGHTS
  --skip
  --save-ctc
  --revcomp
  --recursive
  --quantize
  --no-quantize
  --overlap OVERLAP
  --chunksize CHUNKSIZE
  --batchsize BATCHSIZE
  --max-reads MAX_READS
  -v, --verbose
(bvenv) root@t1v-n-fead2fe2-w-0:/home/test# bonito -v
bonito 0.5.0

The --use_openvino argument does not show up in bonito basecaller -h either. Is this normal, or have I missed something in my install?
@shishir-reddy shishir-reddy changed the title Bonito Basecaller Missing --use_openvino flag Bonito basecaller missing --use_openvino flag Feb 2, 2022
@iiSeymour iiSeymour self-assigned this Feb 2, 2022
@iiSeymour
Copy link
Member

Hey @shishir-reddy

The openvino functionality isn't in the pip package on PyPI currently - it is available on this branch however https://github.com/nanoporetech/bonito/tree/openvino

@shishir-reddy
Copy link
Author

Thanks for the quick response. I switched over to the openvino branch and built with the developer quickstart, but now I am getting another error:

root@t1v-n-fead2fe2-w-0:/home/test# git clone -b openvino https://github.com/nanoporetech/bonito.git
Cloning into 'bonito'...
remote: Enumerating objects: 3370, done.
remote: Counting objects: 100% (1695/1695), done.
remote: Compressing objects: 100% (689/689), done.
remote: Total 3370 (delta 1226), reused 1448 (delta 1002), pack-reused 1675
Receiving objects: 100% (3370/3370), 552.39 KiB | 5.47 MiB/s, done.
Resolving deltas: 100% (2340/2340), done.
root@t1v-n-fead2fe2-w-0:/home/test# ls
bonito
root@t1v-n-fead2fe2-w-0:/home/test# cd bonito/
root@t1v-n-fead2fe2-w-0:/home/test/bonito# python3 -m venv venv3
root@t1v-n-fead2fe2-w-0:/home/test/bonito# source venv3/bin/activate
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip install --upgrade pip
Collecting pip
  Using cached pip-22.0.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-22.0.2
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip install -r requirements.txt 
Collecting mappy==2.24
  Using cached mappy-2.24.tar.gz (140 kB)
  Preparing metadata (setup.py) ... done
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting tqdm==4.31.1
  Using cached tqdm-4.31.1-py2.py3-none-any.whl (48 kB)
Collecting scipy==1.5.4
  Using cached scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl (25.8 MB)
Collecting numpy~=1.19.5
  Using cached numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
Collecting pysam==0.18.0
  Using cached pysam-0.18.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (16.2 MB)
Collecting parasail==1.2
  Using cached parasail-1.2-py2.py3-none-manylinux2010_x86_64.whl (14.1 MB)
Collecting pandas==1.1.5
  Using cached pandas-1.1.5-cp38-cp38-manylinux1_x86_64.whl (9.3 MB)
Collecting requests==2.25.1
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting ont-remora==0.1.1
  Using cached ont_remora-0.1.1-cp38-cp38-linux_x86_64.whl
Collecting ont-fast5-api==3.3.0
  Using cached ont_fast5_api-3.3.0-py3-none-any.whl (2.0 MB)
Collecting fast-ctc-decode==0.3.0
  Using cached fast_ctc_decode-0.3.0-cp38-cp38-manylinux2010_x86_64.whl (508 kB)
Collecting python-dateutil==2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting torch~=1.10.0
  Using cached torch-1.10.2-cp38-cp38-manylinux1_x86_64.whl (881.9 MB)
Collecting koi-cuda102==0.0.5
  Using cached koi_cuda102-0.0.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (350 kB)
Collecting cupy-cuda102==9.6.0
  Using cached cupy_cuda102-9.6.0-cp38-cp38-manylinux1_x86_64.whl (63.0 MB)
Collecting ont-seqdist-cuda102==0.0.4
  Using cached ont_seqdist_cuda102-0.0.4-py3-none-any.whl (21 kB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting pytz>=2017.2
  Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting chardet<5,>=3.0.2
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting onnx
  Using cached onnx-1.10.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.7 MB)
Collecting scikit-learn
  Using cached scikit_learn-1.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB)
Collecting thop
  Using cached thop-0.0.31.post2005241907-py3-none-any.whl (8.7 kB)
Collecting tabulate
  Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting onnxruntime>=1.7
  Using cached onnxruntime-1.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
Collecting h5py>=2.8
  Using cached h5py-3.6.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
Collecting packaging
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting progressbar33>=2.3.1
  Using cached progressbar33-2.4.tar.gz (10 kB)
  Preparing metadata (setup.py) ... done
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting cffi
  Using cached cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)
Collecting fastrlock>=0.5
  Using cached fastrlock-0.8-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (49 kB)
Collecting typing-extensions
  Using cached typing_extensions-4.0.1-py3-none-any.whl (22 kB)
Collecting flatbuffers
  Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting protobuf
  Using cached protobuf-3.19.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB)
Collecting joblib>=0.11
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Using legacy 'setup.py install' for mappy, since package 'wheel' is not installed.
Using legacy 'setup.py install' for progressbar33, since package 'wheel' is not installed.
Installing collected packages: tabulate, pytz, pysam, progressbar33, mappy, flatbuffers, fastrlock, fast-ctc-decode, certifi, wheel, urllib3, typing-extensions, tqdm, toml, threadpoolctl, six, pyparsing, pycparser, protobuf, numpy, joblib, idna, chardet, torch, scipy, requests, python-dateutil, parasail, packaging, onnxruntime, onnx, h5py, cupy-cuda102, cffi, thop, scikit-learn, pandas, ont-seqdist-cuda102, ont-fast5-api, koi-cuda102, ont-remora
  Running setup.py install for progressbar33 ... done
  Running setup.py install for mappy ... done
Successfully installed certifi-2021.10.8 cffi-1.15.0 chardet-4.0.0 cupy-cuda102-9.6.0 fast-ctc-decode-0.3.0 fastrlock-0.8 flatbuffers-2.0 h5py-3.6.0 idna-2.10 joblib-1.1.0 koi-cuda102-0.0.5 mappy-2.24 numpy-1.19.5 onnx-1.10.2 onnxruntime-1.10.0 ont-fast5-api-3.3.0 ont-remora-0.1.1 ont-seqdist-cuda102-0.0.4 packaging-21.3 pandas-1.1.5 parasail-1.2 progressbar33-2.4 protobuf-3.19.4 pycparser-2.21 pyparsing-3.0.7 pysam-0.18.0 python-dateutil-2.8.2 pytz-2021.3 requests-2.25.1 scikit-learn-1.0.2 scipy-1.5.4 six-1.16.0 tabulate-0.8.9 thop-0.0.31.post2005241907 threadpoolctl-3.1.0 toml-0.10.2 torch-1.10.2 tqdm-4.31.1 typing-extensions-4.0.1 urllib3-1.26.8 wheel-0.37.1
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# python setup.py develop
Building with CUDA 10.2
running develop
running egg_info
creating ont_bonito.egg-info
writing ont_bonito.egg-info/PKG-INFO
writing dependency_links to ont_bonito.egg-info/dependency_links.txt
writing entry points to ont_bonito.egg-info/entry_points.txt
writing requirements to ont_bonito.egg-info/requires.txt
writing top-level names to ont_bonito.egg-info/top_level.txt
writing manifest file 'ont_bonito.egg-info/SOURCES.txt'
reading manifest file 'ont_bonito.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ont_bonito.egg-info/SOURCES.txt'
running build_ext
Creating /home/test/bonito/venv3/lib/python3.8/site-packages/ont-bonito.egg-link (link to .)
Adding ont-bonito 0.5.0 to easy-install.pth file
Installing bonito script to /home/test/bonito/venv3/bin

Installed /home/test/bonito
Processing dependencies for ont-bonito==0.5.0
Searching for tqdm==4.31.1
Best match: tqdm 4.31.1
Adding tqdm 4.31.1 to easy-install.pth file
Installing tqdm script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for torch==1.10.2
Best match: torch 1.10.2
Adding torch 1.10.2 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /home/test/bonito/venv3/bin
Installing convert-onnx-to-caffe2 script to /home/test/bonito/venv3/bin
Installing torchrun script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for toml==0.10.2
Best match: toml 0.10.2
Adding toml 0.10.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for scipy==1.5.4
Best match: scipy 1.5.4
Adding scipy 1.5.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for requests==2.25.1
Best match: requests 2.25.1
Adding requests 2.25.1 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for python-dateutil==2.8.2
Best match: python-dateutil 2.8.2
Adding python-dateutil 2.8.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pysam==0.18.0
Best match: pysam 0.18.0
Adding pysam 0.18.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for parasail==1.2
Best match: parasail 1.2
Adding parasail 1.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pandas==1.1.5
Best match: pandas 1.1.5
Adding pandas 1.1.5 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-seqdist-cuda102==0.0.4
Best match: ont-seqdist-cuda102 0.0.4
Adding ont-seqdist-cuda102 0.0.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-remora==0.1.1
Best match: ont-remora 0.1.1
Adding ont-remora 0.1.1 to easy-install.pth file
Installing remora script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-fast5-api==3.3.0
Best match: ont-fast5-api 3.3.0
Adding ont-fast5-api 3.3.0 to easy-install.pth file
Installing check_compression script to /home/test/bonito/venv3/bin
Installing compress_fast5 script to /home/test/bonito/venv3/bin
Installing fast5_subset script to /home/test/bonito/venv3/bin
Installing multi_to_single_fast5 script to /home/test/bonito/venv3/bin
Installing single_to_multi_fast5 script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for numpy==1.19.5
Best match: numpy 1.19.5
Adding numpy 1.19.5 to easy-install.pth file
Installing f2py script to /home/test/bonito/venv3/bin
Installing f2py3 script to /home/test/bonito/venv3/bin
Installing f2py3.8 script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for mappy==2.24
Best match: mappy 2.24
Adding mappy 2.24 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for koi-cuda102==0.0.5
Best match: koi-cuda102 0.0.5
Adding koi-cuda102 0.0.5 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for fast-ctc-decode==0.3.0
Best match: fast-ctc-decode 0.3.0
Adding fast-ctc-decode 0.3.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for cupy-cuda102==9.6.0
Best match: cupy-cuda102 9.6.0
Adding cupy-cuda102 9.6.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for typing-extensions==4.0.1
Best match: typing-extensions 4.0.1
Adding typing-extensions 4.0.1 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for urllib3==1.26.8
Best match: urllib3 1.26.8
Adding urllib3 1.26.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for chardet==4.0.0
Best match: chardet 4.0.0
Adding chardet 4.0.0 to easy-install.pth file
Installing chardetect script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for certifi==2021.10.8
Best match: certifi 2021.10.8
Adding certifi 2021.10.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for idna==2.10
Best match: idna 2.10
Adding idna 2.10 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for six==1.16.0
Best match: six 1.16.0
Adding six 1.16.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for wheel==0.37.1
Best match: wheel 0.37.1
Adding wheel 0.37.1 to easy-install.pth file
Installing wheel script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pytz==2021.3
Best match: pytz 2021.3
Adding pytz 2021.3 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for thop==0.0.31.post2005241907
Best match: thop 0.0.31.post2005241907
Adding thop 0.0.31.post2005241907 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for onnxruntime==1.10.0
Best match: onnxruntime 1.10.0
Adding onnxruntime 1.10.0 to easy-install.pth file
Installing onnxruntime_test script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for onnx==1.10.2
Best match: onnx 1.10.2
Adding onnx 1.10.2 to easy-install.pth file
Installing backend-test-tools script to /home/test/bonito/venv3/bin
Installing check-model script to /home/test/bonito/venv3/bin
Installing check-node script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for scikit-learn==1.0.2
Best match: scikit-learn 1.0.2
Adding scikit-learn 1.0.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for tabulate==0.8.9
Best match: tabulate 0.8.9
Adding tabulate 0.8.9 to easy-install.pth file
Installing tabulate script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for packaging==21.3
Best match: packaging 21.3
Adding packaging 21.3 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for progressbar33==2.4
Best match: progressbar33 2.4
Adding progressbar33 2.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for h5py==3.6.0
Best match: h5py 3.6.0
Adding h5py 3.6.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for cffi==1.15.0
Best match: cffi 1.15.0
Adding cffi 1.15.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for fastrlock==0.8
Best match: fastrlock 0.8
Adding fastrlock 0.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for protobuf==3.19.4
Best match: protobuf 3.19.4
Adding protobuf 3.19.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for flatbuffers==2.0
Best match: flatbuffers 2.0
Adding flatbuffers 2.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for threadpoolctl==3.1.0
Best match: threadpoolctl 3.1.0
Adding threadpoolctl 3.1.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for joblib==1.1.0
Best match: joblib 1.1.0
Adding joblib 1.1.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pyparsing==3.0.7
Best match: pyparsing 3.0.7
Adding pyparsing 3.0.7 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pycparser==2.21
Best match: pycparser 2.21
Adding pycparser 2.21 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Finished processing dependencies for ont-bonito==0.5.0
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip install develop .[openvino]
Processing /home/test/bonito
  Preparing metadata (setup.py) ... done
Collecting develop
  Using cached develop-0.2.1-py3-none-any.whl
Collecting readme-renderer
  Using cached readme_renderer-32.0-py3-none-any.whl (16 kB)
Collecting ruamel.showoutput
  Using cached ruamel.showoutput-0.1.7-py2.py3-none-any.whl (6.1 kB)
Collecting python-hglib
  Using cached python_hglib-2.6.2-py3-none-any.whl
Collecting repo
  Using cached repo-0.1.0-py3-none-any.whl
Collecting configobj
  Using cached configobj-5.0.6-py3-none-any.whl
Requirement already satisfied: cupy-cuda102==9.6.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (9.6.0)
Requirement already satisfied: fast-ctc-decode==0.3.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.3.0)
Requirement already satisfied: koi-cuda102==0.0.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.0.5)
Requirement already satisfied: mappy==2.24 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.24)
Requirement already satisfied: numpy~=1.19.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.19.5)
Requirement already satisfied: ont-fast5-api==3.3.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (3.3.0)
Requirement already satisfied: ont-remora==0.1.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.1.1)
Requirement already satisfied: ont-seqdist-cuda102==0.0.4 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.0.4)
Requirement already satisfied: pandas==1.1.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.1.5)
Requirement already satisfied: parasail==1.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.2)
Requirement already satisfied: pysam==0.18.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.18.0)
Requirement already satisfied: python-dateutil==2.8.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.8.2)
Requirement already satisfied: requests==2.25.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.25.1)
Requirement already satisfied: scipy==1.5.4 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.5.4)
Requirement already satisfied: toml==0.10.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.10.2)
Requirement already satisfied: torch~=1.10.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.10.2)
Requirement already satisfied: tqdm==4.31.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (4.31.1)
Collecting openvino==2021.4.2
  Using cached openvino-2021.4.2-3976-cp38-cp38-manylinux2014_x86_64.whl (28.9 MB)
Requirement already satisfied: fastrlock>=0.5 in ./venv3/lib/python3.8/site-packages (from cupy-cuda102==9.6.0->ont-bonito==0.5.0) (0.8)
Requirement already satisfied: cffi in ./venv3/lib/python3.8/site-packages (from koi-cuda102==0.0.5->ont-bonito==0.5.0) (1.15.0)
Requirement already satisfied: h5py>=2.8 in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (3.6.0)
Requirement already satisfied: packaging in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (21.3)
Requirement already satisfied: progressbar33>=2.3.1 in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (2.4)
Requirement already satisfied: onnx in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.10.2)
Requirement already satisfied: tabulate in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (0.8.9)
Requirement already satisfied: thop in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (0.0.31.post2005241907)
Requirement already satisfied: scikit-learn in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.0.2)
Requirement already satisfied: onnxruntime>=1.7 in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.10.0)
Requirement already satisfied: pytz>=2017.2 in ./venv3/lib/python3.8/site-packages (from pandas==1.1.5->ont-bonito==0.5.0) (2021.3)
Requirement already satisfied: wheel in ./venv3/lib/python3.8/site-packages (from parasail==1.2->ont-bonito==0.5.0) (0.37.1)
Requirement already satisfied: six>=1.5 in ./venv3/lib/python3.8/site-packages (from python-dateutil==2.8.2->ont-bonito==0.5.0) (1.16.0)
Requirement already satisfied: chardet<5,>=3.0.2 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (1.26.8)
Requirement already satisfied: idna<3,>=2.5 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (2021.10.8)
Requirement already satisfied: typing-extensions in ./venv3/lib/python3.8/site-packages (from torch~=1.10.0->ont-bonito==0.5.0) (4.0.1)
Collecting Pygments>=2.5.1
  Using cached Pygments-2.11.2-py3-none-any.whl (1.1 MB)
Collecting bleach>=2.1.0
  Using cached bleach-4.1.0-py2.py3-none-any.whl (157 kB)
Collecting docutils>=0.13.1
  Using cached docutils-0.18.1-py2.py3-none-any.whl (570 kB)
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Requirement already satisfied: flatbuffers in ./venv3/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito==0.5.0) (2.0)
Requirement already satisfied: protobuf in ./venv3/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito==0.5.0) (3.19.4)
Requirement already satisfied: pycparser in ./venv3/lib/python3.8/site-packages (from cffi->koi-cuda102==0.0.5->ont-bonito==0.5.0) (2.21)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./venv3/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito==0.5.0) (3.0.7)
Requirement already satisfied: joblib>=0.11 in ./venv3/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito==0.5.0) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./venv3/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito==0.5.0) (3.1.0)
Building wheels for collected packages: ont-bonito
  Building wheel for ont-bonito (setup.py) ... done
  Created wheel for ont-bonito: filename=ont_bonito-0.5.0-py3-none-any.whl size=56046 sha256=f0e6567fbb10f415d6f3dab93993f5f5f001dc8068b4d84608c9d34148b50b18
  Stored in directory: /tmp/pip-ephem-wheel-cache-j6rskkap/wheels/bb/f9/1f/407faf9f78936492b9157d2c42d87fe723d84b2be4e97f4acf
Successfully built ont-bonito
Installing collected packages: webencodings, ruamel.showoutput, python-hglib, Pygments, openvino, docutils, configobj, repo, bleach, readme-renderer, ont-bonito, develop
  Attempting uninstall: ont-bonito
    Found existing installation: ont-bonito 0.5.0
    Uninstalling ont-bonito-0.5.0:
      Successfully uninstalled ont-bonito-0.5.0
Successfully installed Pygments-2.11.2 bleach-4.1.0 configobj-5.0.6 develop-0.2.1 docutils-0.18.1 ont-bonito-0.5.0 openvino-2021.4.2 python-hglib-2.6.2 readme-renderer-32.0 repo-0.1.0 ruamel.showoutput-0.1.7 webencodings-0.5.1
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# bonito basecaller -h
Traceback (most recent call last):
  File "/home/test/bonito/venv3/bin/bonito", line 5, in <module>
    from bonito import main
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/__init__.py", line 2, in <module>
    from bonito.cli import basecaller, train, evaluate, view, convert, download, export
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 15, in <module>
    import bonito.openvino.basecall
ModuleNotFoundError: No module named 'bonito.openvino'
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# ls bonito/openvino/
basecall.py  model.py

@iiSeymour
Copy link
Member

ah yes, the branch is still a work in progress, you'll also need pip install crf-beam openvino==2021.4.2

@shishir-reddy
Copy link
Author

I am still seeing the same error:

(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip install crf-beam openvino==2021.4.2
Collecting crf-beam
  Downloading crf_beam-0.1.7-cp38-cp38-manylinux2014_x86_64.whl (2.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 66.7 MB/s eta 0:00:00
Requirement already satisfied: openvino==2021.4.2 in ./venv3/lib/python3.8/site-packages (2021.4.2)
Requirement already satisfied: numpy<1.20,>=1.16.6 in ./venv3/lib/python3.8/site-packages (from openvino==2021.4.2) (1.19.5)
Requirement already satisfied: torch~=1.10.0 in ./venv3/lib/python3.8/site-packages (from crf-beam) (1.10.2)
Requirement already satisfied: typing-extensions in ./venv3/lib/python3.8/site-packages (from torch~=1.10.0->crf-beam) (4.0.1)
Installing collected packages: crf-beam
Successfully installed crf-beam-0.1.7
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# bonito basecaller -h
Traceback (most recent call last):
  File "/home/test/bonito/venv3/bin/bonito", line 5, in <module>
    from bonito import main
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/__init__.py", line 2, in <module>
    from bonito.cli import basecaller, train, evaluate, view, convert, download, export
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 15, in <module>
    import bonito.openvino.basecall
ModuleNotFoundError: No module named 'bonito.openvino'
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# ls -l  bonito/openvino/
total 16
-rw-r--r-- 1 root root 1468 Feb  2 10:54 basecall.py
-rw-r--r-- 1 root root 9278 Feb  2 10:54 model.py

It looks like the bonito/openvino directory is not seen by bonito after installation, do I need to point to it somehow?

@iiSeymour
Copy link
Member

Is the virtualenv (venv3) the same one you installed ont-bonito into from PyPI? If so remove ont-bonito and run python setup.py develop.

@shishir-reddy
Copy link
Author

shishir-reddy commented Feb 2, 2022 •
edited
Loading

That is a good call! There was a previous installation, but after uninstalling and rerunning setup.py the problem persists:

(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip uninstall ont-bonito
Found existing installation: ont-bonito 0.5.0
Uninstalling ont-bonito-0.5.0:
  Would remove:
    /home/test/bonito/venv3/bin/bonito
    /home/test/bonito/venv3/lib/python3.8/site-packages/bonito/*
    /home/test/bonito/venv3/lib/python3.8/site-packages/ont_bonito-0.5.0.dist-info/*
Proceed (Y/n)? Y
  Successfully uninstalled ont-bonito-0.5.0
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# python setup.py develop
Building with CUDA 10.2
running develop
running egg_info
writing ont_bonito.egg-info/PKG-INFO
writing dependency_links to ont_bonito.egg-info/dependency_links.txt
writing entry points to ont_bonito.egg-info/entry_points.txt
writing requirements to ont_bonito.egg-info/requires.txt
writing top-level names to ont_bonito.egg-info/top_level.txt
reading manifest file 'ont_bonito.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ont_bonito.egg-info/SOURCES.txt'
running build_ext
Creating /home/test/bonito/venv3/lib/python3.8/site-packages/ont-bonito.egg-link (link to .)
Adding ont-bonito 0.5.0 to easy-install.pth file
Installing bonito script to /home/test/bonito/venv3/bin

Installed /home/test/bonito
Processing dependencies for ont-bonito==0.5.0
Searching for tqdm==4.31.1
Best match: tqdm 4.31.1
Adding tqdm 4.31.1 to easy-install.pth file
Installing tqdm script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for torch==1.10.2
Best match: torch 1.10.2
Adding torch 1.10.2 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /home/test/bonito/venv3/bin
Installing convert-onnx-to-caffe2 script to /home/test/bonito/venv3/bin
Installing torchrun script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for toml==0.10.2
Best match: toml 0.10.2
Adding toml 0.10.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for scipy==1.5.4
Best match: scipy 1.5.4
Adding scipy 1.5.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for requests==2.25.1
Best match: requests 2.25.1
Adding requests 2.25.1 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for python-dateutil==2.8.2
Best match: python-dateutil 2.8.2
Adding python-dateutil 2.8.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pysam==0.18.0
Best match: pysam 0.18.0
Adding pysam 0.18.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for parasail==1.2
Best match: parasail 1.2
Adding parasail 1.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pandas==1.1.5
Best match: pandas 1.1.5
Adding pandas 1.1.5 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-seqdist-cuda102==0.0.4
Best match: ont-seqdist-cuda102 0.0.4
Adding ont-seqdist-cuda102 0.0.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-remora==0.1.1
Best match: ont-remora 0.1.1
Adding ont-remora 0.1.1 to easy-install.pth file
Installing remora script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for ont-fast5-api==3.3.0
Best match: ont-fast5-api 3.3.0
Adding ont-fast5-api 3.3.0 to easy-install.pth file
Installing check_compression script to /home/test/bonito/venv3/bin
Installing compress_fast5 script to /home/test/bonito/venv3/bin
Installing fast5_subset script to /home/test/bonito/venv3/bin
Installing multi_to_single_fast5 script to /home/test/bonito/venv3/bin
Installing single_to_multi_fast5 script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for numpy==1.19.5
Best match: numpy 1.19.5
Adding numpy 1.19.5 to easy-install.pth file
Installing f2py script to /home/test/bonito/venv3/bin
Installing f2py3 script to /home/test/bonito/venv3/bin
Installing f2py3.8 script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for mappy==2.24
Best match: mappy 2.24
Adding mappy 2.24 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for koi-cuda102==0.0.5
Best match: koi-cuda102 0.0.5
Adding koi-cuda102 0.0.5 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for fast-ctc-decode==0.3.0
Best match: fast-ctc-decode 0.3.0
Adding fast-ctc-decode 0.3.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for cupy-cuda102==9.6.0
Best match: cupy-cuda102 9.6.0
Adding cupy-cuda102 9.6.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for typing-extensions==4.0.1
Best match: typing-extensions 4.0.1
Adding typing-extensions 4.0.1 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for chardet==4.0.0
Best match: chardet 4.0.0
Adding chardet 4.0.0 to easy-install.pth file
Installing chardetect script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for idna==2.10
Best match: idna 2.10
Adding idna 2.10 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for certifi==2021.10.8
Best match: certifi 2021.10.8
Adding certifi 2021.10.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for urllib3==1.26.8
Best match: urllib3 1.26.8
Adding urllib3 1.26.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for six==1.16.0
Best match: six 1.16.0
Adding six 1.16.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for wheel==0.37.1
Best match: wheel 0.37.1
Adding wheel 0.37.1 to easy-install.pth file
Installing wheel script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pytz==2021.3
Best match: pytz 2021.3
Adding pytz 2021.3 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for scikit-learn==1.0.2
Best match: scikit-learn 1.0.2
Adding scikit-learn 1.0.2 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for thop==0.0.31.post2005241907
Best match: thop 0.0.31.post2005241907
Adding thop 0.0.31.post2005241907 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for tabulate==0.8.9
Best match: tabulate 0.8.9
Adding tabulate 0.8.9 to easy-install.pth file
Installing tabulate script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for onnxruntime==1.10.0
Best match: onnxruntime 1.10.0
Adding onnxruntime 1.10.0 to easy-install.pth file
Installing onnxruntime_test script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for onnx==1.10.2
Best match: onnx 1.10.2
Adding onnx 1.10.2 to easy-install.pth file
Installing backend-test-tools script to /home/test/bonito/venv3/bin
Installing check-model script to /home/test/bonito/venv3/bin
Installing check-node script to /home/test/bonito/venv3/bin

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for progressbar33==2.4
Best match: progressbar33 2.4
Adding progressbar33 2.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for packaging==21.3
Best match: packaging 21.3
Adding packaging 21.3 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for h5py==3.6.0
Best match: h5py 3.6.0
Adding h5py 3.6.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for cffi==1.15.0
Best match: cffi 1.15.0
Adding cffi 1.15.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for fastrlock==0.8
Best match: fastrlock 0.8
Adding fastrlock 0.8 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for threadpoolctl==3.1.0
Best match: threadpoolctl 3.1.0
Adding threadpoolctl 3.1.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for joblib==1.1.0
Best match: joblib 1.1.0
Adding joblib 1.1.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for flatbuffers==2.0
Best match: flatbuffers 2.0
Adding flatbuffers 2.0 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for protobuf==3.19.4
Best match: protobuf 3.19.4
Adding protobuf 3.19.4 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pyparsing==3.0.7
Best match: pyparsing 3.0.7
Adding pyparsing 3.0.7 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Searching for pycparser==2.21
Best match: pycparser 2.21
Adding pycparser 2.21 to easy-install.pth file

Using /home/test/bonito/venv3/lib/python3.8/site-packages
Finished processing dependencies for ont-bonito==0.5.0
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# pip install develop .[openvino]
Processing /home/test/bonito
  Preparing metadata (setup.py) ... done
Requirement already satisfied: develop in ./venv3/lib/python3.8/site-packages (0.2.1)
Requirement already satisfied: configobj in ./venv3/lib/python3.8/site-packages (from develop) (5.0.6)
Requirement already satisfied: ruamel.showoutput in ./venv3/lib/python3.8/site-packages (from develop) (0.1.7)
Requirement already satisfied: readme-renderer in ./venv3/lib/python3.8/site-packages (from develop) (32.0)
Requirement already satisfied: repo in ./venv3/lib/python3.8/site-packages (from develop) (0.1.0)
Requirement already satisfied: python-hglib in ./venv3/lib/python3.8/site-packages (from develop) (2.6.2)
Requirement already satisfied: cupy-cuda102==9.6.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (9.6.0)
Requirement already satisfied: fast-ctc-decode==0.3.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.3.0)
Requirement already satisfied: koi-cuda102==0.0.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.0.5)
Requirement already satisfied: mappy==2.24 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.24)
Requirement already satisfied: numpy~=1.19.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.19.5)
Requirement already satisfied: ont-fast5-api==3.3.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (3.3.0)
Requirement already satisfied: ont-remora==0.1.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.1.1)
Requirement already satisfied: ont-seqdist-cuda102==0.0.4 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.0.4)
Requirement already satisfied: pandas==1.1.5 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.1.5)
Requirement already satisfied: parasail==1.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.2)
Requirement already satisfied: pysam==0.18.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.18.0)
Requirement already satisfied: python-dateutil==2.8.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.8.2)
Requirement already satisfied: requests==2.25.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2.25.1)
Requirement already satisfied: scipy==1.5.4 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.5.4)
Requirement already satisfied: toml==0.10.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (0.10.2)
Requirement already satisfied: torch~=1.10.0 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (1.10.2)
Requirement already satisfied: tqdm==4.31.1 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (4.31.1)
Requirement already satisfied: openvino==2021.4.2 in ./venv3/lib/python3.8/site-packages (from ont-bonito==0.5.0) (2021.4.2)
Requirement already satisfied: fastrlock>=0.5 in ./venv3/lib/python3.8/site-packages (from cupy-cuda102==9.6.0->ont-bonito==0.5.0) (0.8)
Requirement already satisfied: cffi in ./venv3/lib/python3.8/site-packages (from koi-cuda102==0.0.5->ont-bonito==0.5.0) (1.15.0)
Requirement already satisfied: progressbar33>=2.3.1 in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (2.4)
Requirement already satisfied: h5py>=2.8 in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (3.6.0)
Requirement already satisfied: packaging in ./venv3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito==0.5.0) (21.3)
Requirement already satisfied: thop in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (0.0.31.post2005241907)
Requirement already satisfied: onnx in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.10.2)
Requirement already satisfied: tabulate in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (0.8.9)
Requirement already satisfied: scikit-learn in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.0.2)
Requirement already satisfied: onnxruntime>=1.7 in ./venv3/lib/python3.8/site-packages (from ont-remora==0.1.1->ont-bonito==0.5.0) (1.10.0)
Requirement already satisfied: pytz>=2017.2 in ./venv3/lib/python3.8/site-packages (from pandas==1.1.5->ont-bonito==0.5.0) (2021.3)
Requirement already satisfied: wheel in ./venv3/lib/python3.8/site-packages (from parasail==1.2->ont-bonito==0.5.0) (0.37.1)
Requirement already satisfied: six>=1.5 in ./venv3/lib/python3.8/site-packages (from python-dateutil==2.8.2->ont-bonito==0.5.0) (1.16.0)
Requirement already satisfied: idna<3,>=2.5 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (2021.10.8)
Requirement already satisfied: chardet<5,>=3.0.2 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito==0.5.0) (1.26.8)
Requirement already satisfied: typing-extensions in ./venv3/lib/python3.8/site-packages (from torch~=1.10.0->ont-bonito==0.5.0) (4.0.1)
Requirement already satisfied: Pygments>=2.5.1 in ./venv3/lib/python3.8/site-packages (from readme-renderer->develop) (2.11.2)
Requirement already satisfied: docutils>=0.13.1 in ./venv3/lib/python3.8/site-packages (from readme-renderer->develop) (0.18.1)
Requirement already satisfied: bleach>=2.1.0 in ./venv3/lib/python3.8/site-packages (from readme-renderer->develop) (4.1.0)
Requirement already satisfied: webencodings in ./venv3/lib/python3.8/site-packages (from bleach>=2.1.0->readme-renderer->develop) (0.5.1)
Requirement already satisfied: protobuf in ./venv3/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito==0.5.0) (3.19.4)
Requirement already satisfied: flatbuffers in ./venv3/lib/python3.8/site-packages (from onnxruntime>=1.7->ont-remora==0.1.1->ont-bonito==0.5.0) (2.0)
Requirement already satisfied: pycparser in ./venv3/lib/python3.8/site-packages (from cffi->koi-cuda102==0.0.5->ont-bonito==0.5.0) (2.21)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./venv3/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito==0.5.0) (3.0.7)
Requirement already satisfied: joblib>=0.11 in ./venv3/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito==0.5.0) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./venv3/lib/python3.8/site-packages (from scikit-learn->ont-remora==0.1.1->ont-bonito==0.5.0) (3.1.0)
Building wheels for collected packages: ont-bonito
  Building wheel for ont-bonito (setup.py) ... done
  Created wheel for ont-bonito: filename=ont_bonito-0.5.0-py3-none-any.whl size=56046 sha256=84b9c18a871428018eb439718a7dd5c05de57de6b4de104c6921c7b283ab6571
  Stored in directory: /tmp/pip-ephem-wheel-cache-a656iptj/wheels/bb/f9/1f/407faf9f78936492b9157d2c42d87fe723d84b2be4e97f4acf
Successfully built ont-bonito
Installing collected packages: ont-bonito
  Attempting uninstall: ont-bonito
    Found existing installation: ont-bonito 0.5.0
    Uninstalling ont-bonito-0.5.0:
      Successfully uninstalled ont-bonito-0.5.0
Successfully installed ont-bonito-0.5.0
(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# bonito basecaller -h
Traceback (most recent call last):
  File "/home/test/bonito/venv3/bin/bonito", line 5, in <module>
    from bonito import main
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/__init__.py", line 2, in <module>
    from bonito.cli import basecaller, train, evaluate, view, convert, download, export
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 15, in <module>
    import bonito.openvino.basecall
ModuleNotFoundError: No module named 'bonito.openvino'

There is no installation of ont-bonito outside of the virtual environment as well.

@dkurt
Copy link

dkurt commented Feb 3, 2022

Hi @shishir-reddy! Please add the following empty file: bonito/openvino/__init__.py and just reinstall again:

touch bonito/bonito/openvino/__init__.py

@shishir-reddy
Copy link
Author

Thanks for the help! I am past the ModuleNotFoundError now, but I am instead getting a CuPy error after installation:

(venv3) root@t1v-n-fead2fe2-w-0:/home/test/bonito# bonito -h
Traceback (most recent call last):
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/cupy/__init__.py", line 16, in <module>
    from cupy import _core  # NOQA
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/cupy/_core/__init__.py", line 1, in <module>
    from cupy._core import core  # NOQA
ImportError: libcublas.so.10: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/test/bonito/venv3/bin/bonito", line 5, in <module>
    from bonito import main
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/__init__.py", line 2, in <module>
    from bonito.cli import basecaller, train, evaluate, view, convert, download, export
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 15, in <module>
    import bonito.openvino.basecall
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/openvino/basecall.py", line 3, in <module>
    from bonito.crf.basecall import stitch_results
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/crf/__init__.py", line 1, in <module>
    from .model import Model
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/bonito/crf/model.py", line 9, in <module>
    import seqdist.sparse
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/seqdist/sparse.py", line 10, in <module>
    import cupy as cp
  File "/home/test/bonito/venv3/lib/python3.8/site-packages/cupy/__init__.py", line 37, in <module>
    raise ImportError(_msg) from e
ImportError: CuPy is not correctly installed.

If you are using wheel distribution (cupy-cudaXX), make sure that the version of CuPy you installed matches with the version of CUDA on your host.
Also, confirm that only one CuPy package is installed:
  $ pip freeze

If you are building CuPy from source, please check your environment, uninstall CuPy and reinstall it with:
  $ pip install cupy --no-cache-dir -vvvv

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

original error: libcublas.so.10: cannot open shared object file: No such file or directory

Does the CPU version of Bonito still require CuPy and CUDA to be setup? The VM I am currently running on does not have a GPU. No worries if a GPU is necessary, I am looking to switch to the GPU version of bonito soon anyways.

@dkurt
Copy link

dkurt commented Feb 5, 2022

Please apply this change from #228 (comment). It will enable for CPU only environment.

@shishir-reddy
Copy link
Author

Bonito CPU works for me now after adding the ImportError check, thanks!

@dkurt dkurt mentioned this issue Mar 24, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iiSeymour iiSeymour

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@iiSeymour @dkurt @shishir-reddy