https://blog.hekmon.com Hekmon's blog Mon, 26 May 2025 09:46:28 GMT https://validator.w3.org/feed/docs/rss2.html https://github.com/jpmonette/feed https://blog.hekmon.com/favicon.ico https://blog.hekmon.com All rights reserved Hekmon 2025 https://blog.hekmon.com/blogs/deal-with-deepseek-busy https://blog.hekmon.com/blogs/deal-with-deepseek-busy Sun, 09 Feb 2025 00:00:00 GMT *视频教程：本地部署 DeepSeek-R1* | 工具 | 主要特点 | |:-----|:------------| | **LM Studio** | 一键部署 R1，支持 CPU/GPU | | **Ollama** | 基于终端，适合开发者 | | **SiliconFlow** | 企业级容器化 | #### 2. 中国友好平台  *可在中国大陆访问的平台* | 平台 | 专长 | 免费额度 | |:---------|:----------|:-----------| | [腾讯云](https://cloud.tencent.com/document/product/1772/115969) | 完整 R1/V3 API 访问 | ✔️ 截至 2025-02-25 | | [Metaso Search](https://metaso.cn/) | 联网版 R1 模型 | 无限制 | | [Flowith](https://flowith.io/) | 知识库集成 | 永久免费计划 | #### 3. 全球替代方案  *具有类似 DeepSeek 功能的国际平台* | 服务 | 优势 | 最适合 | |:--------|:---------|:---------| | [Fireworks.ai](https://fireworks.ai/) | R1 API + GPT-4 对等性能 | 开发者 | | [OpenRouter](https://openrouter.ai/) | 多模型网关 | 预算用户 | | [Claude 3](https://www.anthropic.com) | 伦理 AI 护栏 | 研究人员:cite[1]:cite[4] | | [Llama 3](https://ai.meta.com/llama) | 开源定制化 | 注重隐私的用户:cite[1]:cite[10] | --- ### 五大 DeepSeek 替代品对比 #### 1. **Claude 3** - 伦理AI强者 - ✅ 20万token上下文窗口 - ✅ 宪法性AI安全保障 - ❌ 无免费额度 #### 2. **Google Gemini** - 实时数据大师 - ✅ 实时网络数据集成 - ✅ GWorkspace 兼容性 - ❌ 创造力有限 #### 3. **Llama 3** - 隐私卫士 - ✅ 完全开源 - ✅ 本地 GPU 部署 - ❌ 需要编程技能 #### 4. **GitHub Copilot** - 代码专家 - ✅ IDE 集成 - ✅ 支持40+种编程语言 - ❌ 仅限代码领域 #### 5. **Perplexity AI** - 研究优化 - ✅ 支持引用的答案 - ✅ 实时事实核查 - ❌ 无 API 访问 --- ### 企业解决方案矩阵  *企业级 AI 部署选项* | 提供商 | 专业领域 | 合规性 | |:---------|:---------------|:-----------| | 阿里云 | 高容量处理 | 中国国标 | | Azure | 混合云 AI | GDPR 就绪 | | NVIDIA NIM | GPU 优化推理 | 全球认证 | --- ### 保持持续访问的专业建议 1. **收藏** [DeepSeek 状态页面](https://status.deepseek.com) 2. 使用 **VPN 轮换**应对地理封锁 3. 结合**本地 R1 部署**和云端 API 4. 关注 [AI Subreddit](https://reddit.com/r/machinelearning) 获取更新 ]]> wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/deal-with-deepseek-is-busy https://blog.hekmon.com/blogs/deal-with-deepseek-is-busy Sun, 09 Feb 2025 00:00:00 GMT *Video Guide: Deploy DeepSeek-R1 Locally* | Tool | Key Features | |:-----|:------------| | **LM Studio** | 1-click R1 deployment, CPU/GPU support | | **Ollama** | Terminal-based, ideal for developers | | **SiliconFlow** | Enterprise-grade containerization | #### 2. Chinese-Friendly Platforms  *Platforms accessible from mainland China* | Platform | Specialty | Free Tier | |:---------|:----------|:-----------| | [Tencent Cloud](https://cloud.tencent.com/document/product/1772/115969) | Full R1/V3 API access | ✔️ Until 2025-02-25 | | [Metaso Search](https://metaso.cn/) | Web-connected R1 model | Unlimited | | [Flowith](https://flowith.io/) | Knowledge base integration | Permanent free plan | #### 3. Global Alternatives  *International platforms with DeepSeek-like capabilities* | Service | Strength | Best For | |:--------|:---------|:---------| | [Fireworks.ai](https://fireworks.ai/) | R1 API + GPT-4 parity | Developers | | [OpenRouter](https://openrouter.ai/) | Multi-model gateway | Budget users | | [Claude 3](https://www.anthropic.com) | Ethical AI guardrails | Researchers:cite[1]:cite[4] | | [Llama 3](https://ai.meta.com/llama) | Open-source customization | Privacy-focused users:cite[1]:cite[10] | --- ### Top 5 DeepSeek Alternatives Compared #### 1. **Claude 3** - The Ethical Powerhouse - ✅ 200K token context window - ✅ Constitutional AI safeguards - ❌ No free tier #### 2. **Google Gemini** - Real-Time Master - ✅ Live web data integration - ✅ GWorkspace compatibility - ❌ Limited creativity #### 3. **Llama 3** - Privacy Champion - ✅ Fully open-source - ✅ Local GPU deployment - ❌ Requires coding skills #### 4. **GitHub Copilot** - Code Specialist - ✅ IDE integration - ✅ 40+ programming languages - ❌ Code-only focus #### 5. **Perplexity AI** - Research Optimized - ✅ Citation-supported answers - ✅ Real-time fact-checking - ❌ No API access --- ### Enterprise Solutions Matrix  *Enterprise-grade AI deployment options* | Provider | Specialization | Compliance | |:---------|:---------------|:-----------| | Alibaba Cloud | High-volume processing | China GB Standard | | Azure | Hybrid cloud AI | GDPR Ready | | NVIDIA NIM | GPU-optimized inference | Global Certifications | --- ### Pro Tips for Uninterrupted Access 1. **Bookmark** [DeepSeek Status Page](https://status.deepseek.com) 2. Use **VPN rotation** for geo-blocks 3. Combine **local R1 deployment** with cloud APIs 4. Monitor [AI Subreddit](https://reddit.com/r/machinelearning) for updates ]]> wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/deepseek-free-access-web-api https://blog.hekmon.com/blogs/deepseek-free-access-web-api Tue, 18 Feb 2025 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/deepseek-r1-api https://blog.hekmon.com/blogs/deepseek-r1-api Sun, 02 Feb 2025 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/deepseek-r1-full-api https://blog.hekmon.com/blogs/deepseek-r1-full-api Sat, 08 Feb 2025 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/deepseek-to-openai https://blog.hekmon.com/blogs/deepseek-to-openai Fri, 31 Jan 2025 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/develop-plugin-in-win-via-wxt https://blog.hekmon.com/blogs/develop-plugin-in-win-via-wxt Mon, 10 Feb 2025 00:00:00 GMT wxt-react-starter@0.0.0 dev > wxt WXT 0.19.26 4:14:14 PM ✔ Started dev server @ http://localhost:3000 4:14:15 PM ℹ Pre-rendering chrome-mv3 for development with Vite 6.0.8 4:14:15 PM ✔ Built extension in 350 ms 4:14:15 PM ├─ .output/chrome-mv3/manifest.json 1.05 kB ├─ .output/chrome-mv3/popup.html 744 B ├─ .output/chrome-mv3/background.js 19.96 kB ├─ .output/chrome-mv3/chunks/popup-BxzUToPe.js 8.24 kB ├─ .output/chrome-mv3/content-scripts/content.js 38.3 kB ├─ .output/chrome-mv3/icon/128.png 3.07 kB ├─ .output/chrome-mv3/icon/16.png 559 B ├─ .output/chrome-mv3/icon/32.png 916 B ├─ .output/chrome-mv3/icon/48.png 1.33 kB ├─ .output/chrome-mv3/icon/96.png 2.37 kB └─ .output/chrome-mv3/wxt.svg 1.07 kB Σ Total size: 77.61 kB ✖ Command failed after 26.0 s 4:14:40 PM ERROR connect ECONNREFUSED 127.0.0.1:34713 4:14:40 PM at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) ``` ## Windows初尝试 很烦windows上的环境变量配置，习惯了Ubuntu和Mac OS。。现在不得不折腾一下，在vscode中尝试下。 ### 安装node 问了下AI，可以使用Chocolatey来做包管理器，类似linux上的apt和yum，听起来很不错！ 1. 管理员权限在powershell中安装Chocolatey： ``` Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) ``` 2. 安装完Chocolatey后，在powershell中安装node： ``` choco install nodejs ``` 3. 设置环境变量以生效配置 这里有个坑，其实上一步是已经自动配置了环境变量，cmd中已有可以运行`node -v`了，但是，我使用的是powershell！！！不会自动更新环境变量！！！ ``` # powershell中查看环境变量 Get-ChildItem Env:\ ``` 看了下，有网友写了个脚本来生效powershell的环境变量，实测可用： ``` # RefreshEnv.ps1 # # PowerShell脚本用于从注册表读取环境变量并设置会话变量 Write-Host "正在从注册表刷新cmd.exe的环境变量。请稍候..." -NoNewline function Set-FromReg { param ( [string]$regPath, [string]$name, [string]$varName ) $value = Get-ItemProperty -Path $regPath -Name $name -ErrorAction SilentlyContinue if ($value) { Set-Item -Path Env:$varName -Value $value.$name } } function Get-RegEnv { param ( [string]$regPath ) $vars = Get-Item -Path $regPath foreach ($var in $vars.Property) { if ($var -ne "Path") { Set-FromReg $regPath $var $var } } } # 获取系统和用户环境变量 Get-RegEnv "HKLM:\System\CurrentControlSet\Control\Session Manager\Environment" Get-RegEnv "HKCU:\Environment" # 特殊处理PATH - 混合用户和系统路径 $path_HKLM = (Get-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Session Manager\Environment").Path $path_HKCU = (Get-ItemProperty -Path "HKCU:\Environment").Path $env:Path = "$path_HKLM;$path_HKCU" # 保存原始用户名和架构 $OriginalUserName = $env:USERNAME $OriginalArchitecture = $env:PROCESSOR_ARCHITECTURE # 重置用户名和架构 $env:USERNAME = $OriginalUserName $env:PROCESSOR_ARCHITECTURE = $OriginalArchitecture Write-Host "完成" ``` ### wxt插件开发环境 1. 初始化项目，和之前的Windows一样 ``` # 设置成淘宝镜像源 npm config set registry https://registry.npmmirror.com/ npx wxt@latest init myhelper ``` 2. 跟其他一样了。]]> wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/gdb-cross-compile https://blog.hekmon.com/blogs/gdb-cross-compile Sun, 31 Mar 2024 00:00:00 GMT size = SVE_PT_SIZE (AARCH64_MAX_SVE_VQ, SVE_PT_REGS_SVE); | ^~~~~~~~~~~ make[2]: *** [Makefile:546: linux-aarch64-low.o] Error 1 make[1]: *** [Makefile:11607: all-gdbserver] Error 2 make: *** [Makefile:1021: all] Error 2 ``` 经分析，上述错误应该是**c++**编译导致，可加上如下configure选项解决： ``` CXX=${CROSS_COMPILE}g++ ``` 最终可用的配置如下： ``` ./configure --with-mpfr=/usr/local/mpfr-4.2.1 --with-gmp=/usr/local/gmp-6.3.0 --with-mpc=/usr/local/mpc-1.3.1 --host=aarch64-linux-gnu --target=aarch64-linux-gnu CXX=${CROSS_COMPILE}g++ CC=${CROSS_COMPILE}gcc LD=${CROSS_COMPILE}ld AR=${CROSS_COMPILE}ar --prefix=xxx/gdb-14.1/ ``` # 2.2 编译产物 解决完上述问题后，最终编译出可在样机中执行的gdbserver # 2.3 gdbserver使用 ## 2.3.1 **样机端** 1. 将上述附件中的**gdbserver_stripped** tftp到样机的/bin目录下： ``` cd /bin;tftp -gr gdbserver 192.168.1.105;chmod +x gdbserver_stripped; ``` 2. 运行gdbserver: ``` gdbserver_stripped 192.168.1.105:1234 --attach `pgrep main` ``` ## 2.3.2 **主机端** 1. 运行toolchain的gdb: ``` aarch64-linux-gnu-gdb # 注意提前设置环境变量 ``` 2. 远程连接样机： ``` set sysroot xxx target remote 192.168.1.76:1234 ``` 3. 样机串口打印如下信息说明**gdbserver已成功连接：** ``` Attached; pid = 1300 gdbserver: Unable to determine the number of hardware watchpoints available. gdbserver: Unable to determine the number of hardware breakpoints available. Listening on port 1234 Remote debugging from host 192.168.1.105, port 3465 ``` 4. 在主机端的gdb中执行如下指令保存样机当前的core信息： ``` gcore ./core_dump.test #或使用generate-core-file ./core_dump.test指令 ``` 执行上述指令后，可在当前路径下生成core文件**core_dump.test**，如下所示： ``` file core_dump.test core_dump.test: ELF 64-bit LSB core file, ARM aarch64, version 1 (SYSV), SVR4-style, from '/bin/main' ``` 接下来就可以使用gdb来分析上述core_dump.test了。]]> wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/nothing-but-everything-2025-05 https://blog.hekmon.com/blogs/nothing-but-everything-2025-05 Sun, 25 May 2025 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/oversea-sim https://blog.hekmon.com/blogs/oversea-sim Mon, 29 Apr 2024 00:00:00 GMT wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/run-tensorflow-in-arm https://blog.hekmon.com/blogs/run-tensorflow-in-arm Fri, 14 Feb 2025 00:00:00 GMT > /etc/opkg.conf echo arch aarch64_generic 200 >> /etc/opkg.conf echo arch aarch64_cortex-a73_neon-vfpv4 300 >> /etc/opkg.conf ``` This number refers to the priority of the software source. ### Install Python and pip (to memory) First, modify the environment variable for dynamic libraries: ``` # Execute on DUT export LD_LIBRARY_PATH=/tmp/usr/lib:$LD_LIBRARY_PATH alias pip='python -m pip' ``` And define an alias for pip. If you install pip this way, without defining an alias, you can only execute it through `python -m pip`. #### OPKG Automated Offline Installation Since the DUT does not have `wget`, and my previous attempt to install `wget` failed, we cannot use `opkg install` for **online installation** on the DUT. Initially, my idea was to directly download the required packages from the mirror source and then **install them offline**. However, this approach has many disadvantages: 1. You need to download dependency packages one by one, and you only know the dependencies after installation fails. 2. Changing versions or architectures requires starting over, which is very troublesome. My current solution is to write a script to resolve opkg dependencies, download the dependency packages on the PC, generate a shell script with the dependency relationships, and then transfer it to the DUT for **offline installation**. (Dependency relationships are resolved by parsing the Packages file under each category.) The entire process is very automated. Just provide the required opkg names, TFTP server address, mirror source, and a few configuration options. This Python script will download the required packages, dependency packages, generate dependency relationships, and even automatically generate the DUT installation script. #### Automated Python and tflite_runtime Environment Integration Script For the integration of Python and tflite_runtime, an automated Python script has also been implemented: `tflite_runtime_py_env_prepare.py`, as follows: ``` import argparse import os import subprocess OPKGS_NEEDED = [ "python3-pip", "python3-numpy", "python3-pillow", # "gcc", ] TFTP_SERVER_IP = "192.168.1.100" OPKG_MIRROR_URL = "https://mirrors.aliyun.com/openwrt/releases/packages-23.05/aarch64_generic/" # Precise to version and architecture INSTALL_IN_RAM = True # Whether to install in RAM def shell_cmd_pkg(url_prefix, pkg_name): global INSTALL_IN_RAM shell_cmd = "" if url_prefix: os.system(f"wget --no-check-certificate --no-clobber {url_prefix}{pkg_name}") shell_cmd += f"tftp -gr {pkg_name} {TFTP_SERVER_IP} && " shell_cmd += f"chmod 777 {pkg_name} && " if pkg_name.endswith(".ipk"): shell_cmd += f"opkg install {pkg_name}{' -d ram' if INSTALL_IN_RAM else ''} && " elif pkg_name.endswith(".sh"): shell_cmd += f"./{pkg_name} && " elif pkg_name.endswith(".whl"): shell_cmd += f"python -m pip install {pkg_name} && " shell_cmd += f"rm {pkg_name}\n" return shell_cmd if __name__ == '__main__': shell_cmd_content = "" # Pre-preparation shell_cmd_content += "cd /tmp\n" shell_cmd_content += "export LD_LIBRARY_PATH=/tmp/usr/lib:$LD_LIBRARY_PATH\n" # for gcc, aarch64 # shell_cmd_content += shell_cmd_pkg("https://downloads.openwrt.org/snapshots/targets/ipq807x/generic/packages/", "libstdcpp6_12.3.0-4_aarch64_cortex-a53.ipk") # for py311, armhf # shell_cmd_content += shell_cmd_pkg("https://mirrors.aliyun.com/openwrt/releases/23.05.5/targets/mediatek/mt7629/packages/", "libatomic1_12.3.0-4_arm_cortex-a7.ipk") # Install Python and pip subprocess.run(["python3", "opkg_prepare.py", "--tftp-server-ip", TFTP_SERVER_IP, "--mirror-url", OPKG_MIRROR_URL,] + (["--ram"] if INSTALL_IN_RAM else []) + OPKGS_NEEDED) shell_cmd_content += shell_cmd_pkg(None, "opkg_install_from_tftp.sh") # Install pip # os.system("wget --no-check-certificate -nc https://bootstrap.pypa.io/pip/get-pip.py") # shell_cmd_content += f"tftp -gr get-pip.py {TFTP_SERVER_IP} && python3 get-pip.py && rm get-pip.py\n" # Install pkginfo shell_cmd_content += shell_cmd_pkg("https://files.pythonhosted.org/packages/c0/38/d617739840a2f576e400f03fea0a # Install tflite_runtime shell_cmd_content += shell_cmd_pkg("xxx", "tflite_runtime-2.18.0-cp311-cp311-linux_aarch64.whl") with open("DUT_env_prepare.sh", "w") as file: file.write(shell_cmd_content) ``` (WARNING) However, for tflite-runtime, it currently only supports pairing with python3.11 (i.e., packages-23.05), due to the numpy version requirement of tensorflow-2.18.0, only packages-23.05 is compatible in the official source and openwrt source. Based on your own environment and needs, modify **OPKGS_NEEDED, TFTP_SERVER_IP, OPKG_MIRROR_URL, INSTALL_IN_RAM** in tflite_runtime_py_env_prepare.py. After executing tflite_runtime_py_env_prepare.py on the PC, you will get all dependencies and a file named **DUT_env_prepare.sh**. Transfer this sh file to the DUT and run it to achieve offline installation of tflite_runtime. If you only need to integrate python, simply comment out the part containing `"tflite_runtime-2.18.0-cp311-cp311-linux_aarch64.whl"` in tflite_runtime_py_env_prepare.py. ### Verify tflite Integration Success If the output is as follows, it means the integration is successful. ``` INFO: Created TensorFlow Lite XNNPACK delegate for CPU. Prediction: tabby, tabby cat ``` # Locally Build tflite-runtime Wheel Package The official only provides the glibc wheel package, but openwrt's C standard library is musl, so we need to rebuild it ourselves. During the build process, remember to refer to these URLs: [kasitoru/tflite_runtime-musl: Build the TensorFlow Lite tflite_runtime Python library for ARM devices using musl libc (github.com)](https://github.com/kasitoru/tflite_runtime-musl/tree/main) [feranick/TFlite-builds: TFlite cross-platform builds (github.com)](https://github.com/feranick/TFlite-builds) Here is a record of the build process and the pitfalls encountered: ## tensorflow Download First, you need to download tensorflow from github. It's almost impossible to git clone successfully, so we can select a fixed version from the tag and download the zip package. ## Cross-compilation Based on musl libc Quick build: 1. cd to your tensorflow-2.18-rc1 folder. 2. Download the patch file from T417309#10451504. 3. `patch -p1 < tflite-2-18-rc1-musl.patch` 4. `make -C tensorflow/lite/tools/pip_package docker-build TENSORFLOW_TARGET=aarch64 PYTHON_VERSION=3.11` 5. After successfully building the docker, execute: `/tensorflow/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh $(TENSORFLOW_TARGET)` 6. Wait for the build to complete. ### Docker Network Unblocking Most Docker build failures are due to network issues, such as the ubuntu source (amd64), ubuntu-ports source (arm64 and armhf), and PPA source. One reason is the intranet gateway, and the other is the inability to access foreign sources. This description is concise, but this pitfall actually took me a lot of time. Things to note: 1. Dockerfile.py3 1. Both wget and curl https links need to cancel authentication. 2. Adding deadsnakes' ppa will cause errors, so switch to adding mirror sources. 3. Add timezone when apt-get install, otherwise it will get stuck. 4. Modify cmake version, refer to https://github.com/feranick/TFlite-builds 2. update_sources.sh and update_ppa_deadsnakes.sh: change sources 1. Cannot use http sources, otherwise `hash sum mismatch` error will occur, only https sources can be used. 2. Use USTC mirror for PPA https://launchpad.proxy.ustclug.org/ 3. Makefile: refer to https://github.com/feranick/TFlite-builds All modifications are commented, see the diff file below for details: ```diff diff --git a/tensorflow/lite/tools/pip_package/Dockerfile.py3 b/tensorflow/lite/tools/pip_package/Dockerfile.py3 index 63373905..d9035768 100644 --- a/tensorflow/lite/tools/pip_package/Dockerfile.py3 +++ b/tensorflow/lite/tools/pip_package/Dockerfile.py3 @@ -33,33 +33,40 @@ RUN apt-get update && \ apt-get clean # Install Bazel. -RUN wget https://github.com/bazelbuild/bazelisk/releases/download/v1.15.0/bazelisk-linux-amd64 \ +# wget and curl https links need to cancel authentication. +RUN wget --no-check-certificate https://github.com/bazelbuild/bazelisk/releases/download/v1.15.0/bazelisk-linux-amd64 \ -O /usr/local/bin/bazel && chmod +x /usr/local/bin/bazel # Install Python packages. RUN dpkg --add-architecture armhf RUN dpkg --add-architecture arm64 -RUN yes | add-apt-repository ppa:deadsnakes/ppa -RUN apt-get update && \ - apt-get install -y \ +## Adding deadsnakes' ppa will cause errors, so switch to adding mirror sources. +COPY update_ppa_deadsnakes.sh / +RUN /update_ppa_deadsnakes.sh +# RUN yes | add-apt-repository ppa:deadsnakes/ppa +RUN apt-get update +# Add timezone when apt-get install, otherwise it will get stuck. +RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC \ + apt-get install -y \ python$PYTHON_VERSION \ python$PYTHON_VERSION-dev \ python$PYTHON_VERSION-venv \ python$PYTHON_VERSION-distutils \ libpython$PYTHON_VERSION-dev \ libpython$PYTHON_VERSION-dev:armhf \ - libpython$PYTHON_VERSION-dev:arm64 + libpython$PYTHON_VERSION-dev:arm64 RUN ln -sf /usr/bin/python$PYTHON_VERSION /usr/bin/python3 -RUN curl -OL https://bootstrap.pypa.io/get-pip.py +RUN curl -k -OL https://bootstrap.pypa.io/get-pip.py RUN python3 get-pip.py RUN rm get-pip.py RUN pip3 install --upgrade pip RUN pip3 install numpy~=$NUMPY_VERSION setuptools pybind11 RUN ln -sf /usr/include/python$PYTHON_VERSION /usr/include/python3 RUN ln -sf /usr/local/lib/python$PYTHON_VERSION/dist-packages/numpy/core/include/numpy /usr/include/python3/numpy -RUN curl -OL https://github.com/Kitware/CMake/releases/download/v3.16.8/cmake-3.16.8-Linux-x86_64.sh +# Modify cmake version, refer to https://github.com/feranick/TFlite-builds +RUN curl -k -OL https://cmake.org/files/v3.29/cmake-3.29.6-linux-x86_64.sh RUN mkdir /opt/cmake -RUN sh cmake-3.16.8-Linux-x86_64.sh --prefix=/opt/cmake --skip-license +RUN sh cmake-3.29.6-linux-x86_64.sh --prefix=/opt/cmake --skip-license RUN ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake ENV CI_BUILD_PYTHON=python$PYTHON_VERSION diff --git a/tensorflow/lite/tools/pip_package/Makefile b/tensorflow/lite/tools/pip_package/Makefile index 24bc4970..ff85b8da 100644 --- a/tensorflow/lite/tools/pip_package/Makefile +++ b/tensorflow/lite/tools/pip_package/Makefile @@ -13,9 +13,10 @@ # limitations under the License. # Values: debian:, ubuntu: BASE_IMAGE ?= ubuntu:20.04 PYTHON_VERSION ?= 3.11 -NUMPY_VERSION ?= 1.23.2 +NUMPY_VERSION ?= 1.24.4 # Values: rpi, aarch64, native TENSORFLOW_TARGET ?= native @@ -73,4 +74,7 @@ docker-build: docker-image --rm --interactive $(shell tty -s && echo --tty) \ $(DOCKER_PARAMS) \ $(TAG_IMAGE) \ - /with_the_same_user /bin/bash -C /tensorflow/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh $(TENSORFLOW_TARGET) + /with_the_same_user /bin/bash + # /with_the_same_user /bin/bash -C /tensorflow/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh $(TENSORFLOW_TARGET) +# Slightly modified above to avoid reloading docker every time there is a compilation error +# Correspondingly, after entering the container, manually execute the command after the commented-out bash -C diff --git a/tensorflow/lite/tools/pip_package/update_ppa_deadsnakes.sh b/tensorflow/lite/tools/pip_package/update_ppa_deadsnakes.sh new file mode 100755 index 00000000..0332e665 --- /dev/null +++ b/tensorflow/lite/tools/pip_package/update_ppa_deadsnakes.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +set -ex + +. /etc/os-release + +[[ "${NAME}" == "Ubuntu" ]] || exit 0 + +yes | apt-get install gnupg +apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA6932366A755776 # USTC PPA source public key + +cat <> /etc/apt/sources.list + +## python deadsnakes ppa +deb https://launchpad.proxy.ustclug.org/deadsnakes/ppa/ubuntu/ ${UBUNTU_CODENAME} main +# deb-src https://launchpad.proxy.ustclug.org/deadsnakes/ppa/ubuntu/ ${UBUNTU_CODENAME} main + +EOT \ No newline at end of file diff --git a/tensorflow/lite/tools/pip_package/update_sources.sh b/tensorflow/lite/tools/pip_package/update_sources.sh index 40e3213c..bf4898e5 100755 --- a/tensorflow/lite/tools/pip_package/update_sources.sh +++ b/tensorflow/lite/tools/pip_package/update_sources.sh @@ -15,14 +15,41 @@ # ============================================================================== #!/bin/bash + +set -ex + . /etc/os-release [[ "${NAME}" == "Ubuntu" ]] || exit 0 sed -i "s/deb\ /deb \[arch=amd64\]\ /g" /etc/apt/sources.list + +apt-get update + +# Without installing ca-certificates, https cannot be downloaded +yes | apt-get install ca-certificates + cat <> /etc/apt/sources.list -deb [arch=arm64,armhf] http://ports.ubuntu.com/ubuntu-ports ${UBUNTU_CODENAME} main universe -deb [arch=arm64,armhf] http://ports.ubuntu.com/ubuntu-ports ${UBUNTU_CODENAME}-updates main universe -deb [arch=arm64,armhf] http://ports.ubuntu.com/ubuntu-ports ${UBUNTU_CODENAME}-security main universe + +## aarch64 and armhf sources + +deb [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME} main restricted universe multiverse +# deb-src [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME} main restricted universe multiverse + +deb [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-security main restricted universe multiverse +# deb-src [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-security main restricted universe multiverse + +deb [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-updates main restricted universe multiverse +# deb-src [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-updates main restricted universe multiverse + +deb [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-backports main restricted universe multiverse +# deb-src [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-backports main restricted universe multiverse + +## Not recommended +# deb [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-proposed main restricted universe multiverse +# deb-src [arch=arm64,armhf] https://repo.huaweicloud.com/ubuntu-ports/ ${UBUNTU_CODENAME}-proposed main restricted universe multiverse + EOT ``` ### Cross-compilation based on musl libc Reference: [kasitoru/tflite_runtime-musl: Build the TensorFlow Lite tflite_runtime Python library for ARM devices using musl libc (github.com)](https://github.com/kasitoru/tflite_runtime-musl/tree/main) [Build TensorFlow Lite Python Wheel Package (google.cn)](https://tensorflow.google.cn/lite/guide/build_cmake_pip?hl=zh-cn) Things to note: All modifications are commented, see the diff file below for details: 1. download_toolchains.sh 1. Change the toolchain from glibc to musl libc 2. The toolchain cannot be downloaded from external URLs, I have replaced it, and the toolchain files are available on the intranet pha. 3. If this script is not modified or fails after modification, the error `error: conflicting types for 'cpuinfo_isa'; have 'struct cpuinfo_arm_isa'` will occur 2. build_pip_package_with_cmake.sh 1. Disable verification and increase the buffer size for successful git clone 2. [Apply a patch, otherwise compilation will fail based on musl](https://github.com/sartura/flatbuffers/commit/92bd62407329caacd66e92e5bfd2949f2f137bfe#diff-cbd47ef1c2023c38eaa5f2ae941fb09e74c38cafdfbbf968ea68b6cc96a7d257R268) ```diff diff --git a/tensorflow/lite/tools/cmake/download_toolchains.sh b/tensorflow/lite/tools/cmake/download_toolchains.sh index 02ff70c7..9ea3ad6c 100755 --- a/tensorflow/lite/tools/cmake/download_toolchains.sh +++ b/tensorflow/lite/tools/cmake/download_toolchains.sh @@ -14,13 +14,9 @@ # limitations under the License. # ============================================================================== -# Download GCC 8.3 based toolchains. -# Using up-to-date toolchain introduces compatibility issues. -# https://github.com/tensorflow/tensorflow/issues/59631 -# -# In Bazel build, we uses GCC 11.3 based toolchains to support FP16 better -# with XNNPACK. https://github.com/tensorflow/tensorflow/pull/57585 - +# If this script is not modified or fails after modification, the error +# error: conflicting types for 'cpuinfo_isa'; have 'struct cpuinfo_arm_isa' +# will occur set -e SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -30,55 +26,88 @@ TOOLCHAINS_DIR=$(realpath tensorflow/lite/tools/cmake/toolchains) mkdir -p ${TOOLCHAINS_DIR} case $1 in - armhf) - if [[ ! -d "${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf" ]]; then - curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/bin rel/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz >&2 - tar xvf gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C ${TOOLCHAINS_DIR} >&2 + armhf) + ARMCC_ROOT=${TOOLCHAINS_DIR}/armv7l-linux-musleabihf-cross + if [[ ! -d ${ARMCC_ROOT} ]]; then + curl -LO https://more.musl.cc/10/x86_64-linux-musl/armv7l-linux-musleabihf-cross.tgz >&2 + tar zxvf armv7l-linux-musleabihf-cross.tgz -C ${TOOLCHAINS_DIR} >&2 + rm armv7l-linux-musleabihf-cross.tgz + echo '#define __BEGIN_DECLS extern "C" {' >> "${ARMCC_ROOT}/armv7l-linux-musleabihf/include/features.h" + echo '#define __END_DECLS }' >> "${ARMCC_ROOT}/armv7l-linux-musleabihf/include/features.h" + echo '#define __THROW' >> "${ARMCC_ROOT}/armv7l-linux-musleabihf/include/features.h" + echo '#define __nonnull(params)' >> "${ARMCC_ROOT}/armv7l-linux-musleabihf/include/features.h" fi - ARMCC_ROOT=${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf echo "ARMCC_FLAGS=\"-march=armv7-a -mfpu=neon-vfpv4 -funsafe-math-optimizations \ - -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include \ - -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include-fixed \ - -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/include/c++/8.3.0 \ - -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/libc/usr/include \ + -isystem ${ARMCC_ROOT}/armv7l-linux-musleabihf/include/c++/10.2.1 \ + -isystem ${ARMCC_ROOT}/armv7l-linux-musleabihf/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/armv7l-linux-musleabihf/10.2.1/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/armv7l-linux-musleabihf/10.2.1/include-fixed \ -isystem \"\${CROSSTOOL_PYTHON_INCLUDE_PATH}\" \ -isystem /usr/include\"" - echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/arm-linux-gnueabihf-" - ;; - aarch64) - if [[ ! -d "${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu" ]]; then - curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz >&2 - tar xvf gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz -C ${TOOLCHAINS_DIR} >&2 + echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/armv7l-linux-musleabihf-" + ;; + aarch64) + ARMCC_ROOT=${TOOLCHAINS_DIR}/aarch64-linux-musl-cross + if [[ ! -d ${ARMCC_ROOT} ]]; then + curl -LO https://more.musl.cc/10/x86_64-linux-musl/aarch64-linux-musl-cross.tgz >&2 + tar zxvf aarch64-linux-musl-cross.tgz -C ${TOOLCHAINS_DIR} >&2 + rm aarch64-linux-musl-cross.tgz fi - ARMCC_ROOT=${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu echo "ARMCC_FLAGS=\"-funsafe-math-optimizations \ - -isystem ${ARMCC_ROOT}/lib/gcc/aarch64-linux-gnu/8.3.0/include \ - -isystem ${ARMCC_ROOT}/lib/gcc/aarch64-linux-gnu/8.3.0/include-fixed \ - -isystem ${ARMCC_ROOT}/aarch64-linux-gnu/include/c++/8.3.0 \ - -isystem ${ARMCC_ROOT}/aarch64-linux-gnu/libc/usr/include \ + -isystem ${ARMCC_ROOT}/aarch64-linux-musl/include/c++/10.2.1 \ + -isystem ${ARMCC_ROOT}/aarch64-linux-musl/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/aarch64-linux-musl/10.2.1/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/aarch64-linux-musl/10.2.1/include-fixed \ -isystem \"\${CROSSTOOL_PYTHON_INCLUDE_PATH}\" \ -isystem /usr/include\"" - echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/aarch64-linux-gnu-" - ;; - rpi0) - if [[ ! -d "${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf" ]]; then - curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz >&2 - tar xvf gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C ${TOOLCHAINS_DIR} >&2 + echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/aarch64-linux-musl-" + ;; + rpi0) + ARMCC_ROOT=${TOOLCHAINS_DIR}/armv6-linux-musleabihf-cross + if [[ ! -d ${ARMCC_ROOT} ]]; then + curl -LO https://more.musl.cc/10/x86_64-linux-musl/armv6-linux-musleabihf-cross.tgz >&2 + tar zxvf armv6-linux-musleabihf-cross.tgz -C ${TOOLCHAINS_DIR} >&2 + rm armv6-linux-musleabihf-cross.tgz + echo '#define __BEGIN_DECLS extern "C" {' >> "${ARMCC_ROOT}/armv6-linux-musleabihf/include/features.h" + echo '#define __END_DECLS }' >> "${ARMCC_ROOT}/armv6-linux-musleabihf/include/features.h" + echo '#define __THROW' >> "${ARMCC_ROOT}/armv6-linux-musleabihf/include/features.h" + echo '#define __nonnull(params)' >> "${ARMCC_ROOT}/armv6-linux-musleabihf/include/features.h" fi - ARMCC_ROOT=${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf - echo "ARMCC_FLAGS=\"-march=armv6 -mfpu=vfp -mfloat-abi=hard -funsafe-math-optimizations \ - -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include \ - -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include-fixed \ - -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/include/c++/8.3.0 \ - -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/libc/usr/include \ + echo "ARMCC_FLAGS=\"-march=armv6 -mfpu=vfp -funsafe-math-optimizations \ + -isystem ${ARMCC_ROOT}/armv6-linux-musleabihf/include/c++/10.2.1 \ + -isystem ${ARMCC_ROOT}/armv6-linux-musleabihf/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/armv6-linux-musleabihf/10.2.1/include \ + -isystem ${ARMCC_ROOT}/lib/gcc/armv6-linux-musleabihf/10.2.1/include-fixed \ -isystem \"\${CROSSTOOL_PYTHON_INCLUDE_PATH}\" \ -isystem /usr/include\"" - echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/arm-linux-gnueabihf-" - ;; - *) - echo "Usage: download_toolchains.sh [armhf|aarch64|rpi0]" >&2 + echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/armv6-linux-musleabihf-" + ;; + *) + echo "Usage: download_toolchains.sh [armhf|aarch64|rpi0]" >&2 exit - ;; + ;; esac echo "download_toolchains.sh completed successfully." >&2 diff --git a/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh b/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh index aa5b9eb7..6b453575 100755 --- a/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh +++ b/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh @@ -15,6 +15,12 @@ # ============================================================================== set -ex +# For the subsequent git clone to succeed +git config --global http.sslverify false +git config --global https.sslverify false +git config --global http.postBuffer 52428000 +git config --global https.postBuffer 52428000 + SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PYTHON="${CI_BUILD_PYTHON:-python3}" VERSION_SUFFIX=${VERSION_SUFFIX:-} @@ -50,7 +56,9 @@ if [ ! -z "${CI_BUILD_HOME}" ] && [ `pwd` = "/workspace" ]; then fi # Build source tree. -rm -rf "${BUILD_DIR}" && mkdir -p "${BUILD_DIR}/tflite_runtime" +# Do not delete it every time you compile, change it to manual deletion. Otherwise, you have to re-download the dependency packages every time you compile, which is very slow. +# rm -rf "${BUILD_DIR}" && mkdir -p "${BUILD_DIR}/tflite_runtime" +mkdir -p "${BUILD_DIR}/tflite_runtime" cp -r "${TENSORFLOW_LITE_DIR}/tools/pip_package/debian" \ "${TENSORFLOW_LITE_DIR}/tools/pip_package/MANIFEST.in" \ "${TENSORFLOW_LITE_DIR}/python/interpreter_wrapper" \ @@ -120,7 +128,7 @@ case "${TENSORFLOW_TARGET}" in -DCMAKE_SYSTEM_PROCESSOR=aarch64 \ -DXNNPACK_ENABLE_ARM_I8MM=OFF \ -DTFLITE_HOST_TOOLS_DIR="${HOST_BUILD_DIR}" \ - "${TENSORFLOW_LITE_DIR}" + "${TENSORFLOW_LITE_DIR}" --debug-output # If the compilation process is not detailed, often one of the packages gets stuck due to network issues, and it remains stuck for a long time without knowing. ;; native) BUILD_FLAGS=${BUILD_FLAGS:-"-march=native -I${PYTHON_INCLUDE} -I${PYBIND11_INCLUDE} -I${NUMPY_INCLUDE}"} @@ -138,6 +146,13 @@ case "${TENSORFLOW_TARGET}" in ;; esac +# refer https://github.com/versatica/mediasoup/issues/1223 +# refer https://github.com/sartura/flatbuffers/commit/92bd62407329caacd66e92e5bfd2949f2f137bfe#diff-cbd47ef1c2023c38eaa5f2ae941fb09e74c38cafdfbbf968ea68b6cc96a7d257R268 +cd "${BUILD_DIR}/cmake_build/flatbuffers/include/flatbuffers" +patch base.h < flatbuffers_base.h.diff +cd - + cmake --build . --verbose -j ${BUILD_NUM_JOBS} -t _pywrap_tensorflow_interpreter_wrapper cd "${BUILD_DIR}" ``` ]]> wishhself@gmail.com (Hekmon) https://blog.hekmon.com/blogs/travel-to-HongKong-2024 https://blog.hekmon.com/blogs/travel-to-HongKong-2024 Wed, 01 May 2024 00:00:00 GMT wishhself@gmail.com (Hekmon)