feat(net): 合并主线 && 实现unix stream pairs (#944)

* 修复mprotect系统调用未正确设置vm_flags的错误 (#847)

* fix(time): modify update wall time (#836)

更改了时间子系统的update_wall_time函数，通过读取当前周期数，计算delta值进行更新，而不是通过传入delta值进行更新

* chore: 调整triagebot.toml以适应新的组织架构 (#848)

* doc: 完善README.md (#849)

* doc: 完善README.md

* chore: 更新sphinx相关配置，适应read the docs的更新 (#850)

根据read the docs在7月15日blog,进行此修改

https://about.readthedocs.com/blog/2024/07/addons-by-default/

* feat(driver/net): 实现Loopback网卡接口 (#845)

* 初步实现loopback设备

* fix: build-scripts和tools目录下的make check指定工具链版本 (#861)

* fix: tcp poll没有正确处理posix socket的listen状态的问题 (#859)

* chore: 将工具链更新到2024-07-23 (#864)

* chore: 将工具链更新到2024-07-23

* feat(fs): add eventfd syscall support (#858)

* feat(fs): add eventfd syscall support

* refactor: 删除过时的va-pa转换函数，改为统一使用MMArch (#862)

* 默认nightly-2024-07-23 & config改为config.toml (#872)

* fix: 修复由于升级到2024-07-23工具链之后，某些机器上面内核运行一直fault的问题。 (#870)

* fix: 修复由于升级到2024-07-23工具链之后，某些机器上面内核运行一直fault的问题。

* feat(cred): 初步实现Cred (#846)

* 初步实现Cred

* 添加seteuid和setegid

* 添加cred测试程序

* 修改Cred::fscmp返回结果为CredFsCmp枚举

* 完善root用户相关信息

* fix: 修复键盘码解析器没能正确处理类似ctrl C的控制字符的问题 (#877)

* fix: 修复键盘码解析器没能正确处理类似ctrl C的控制字符的问题

* fix: 解决ntty潜在的panic问题

* ci: enable ci workflow on branches other than master (#891)

* 修复unlink、unlinkat系统调用的路径错误 (#892)

* fix: socket shutdown wrong implement (#893)

* feat: 增加tokio异步运行时支持 (#894)

* fix the EventFdFlags error

* feat: support tokio (Single thread version)

Fix deadlock issue on closing file.
Add function for PipeInode and EventFdInode.

* fix: pipe 读取/写入阻塞时,无法kill进程的问题 (#889)

* fix: 修复存在多个virtio设备时，中断号冲突的问题 (#904)

* feat: 添加gendisk抽象 (#903)

* feat: 添加gendisk抽象.

* 支持使用virtio磁盘作为根文件系统

* Update initial_kthread.rs to resolve conflict.

---------

Co-authored-by: Donkey Kane <[email protected]>

* feat(mm): 简单实现fat文件系统的文件映射 (#840)

- 添加文件映射相关接口，目前已简单实现fat文件系统的私有映射和共享映射
- 添加msync系统调用（由于当前未实现脏页自动回写，需要手动调用msync进行同步）
- 简单实现PageCache（暂时使用HashMap进行文件页号与页的映射）
- 添加新的PageFlags标志结构，原PageFlags改名为EntryFlags
- 参考linux使用protection_map映射表进行页面标志的获取
- 添加页面回收机制
- 添加页面回收内核线程
- 缺页中断使用的锁修改为irq_save; 添加脏页回写机制
- 修复do_cow_page死锁问题
- 访问非法地址时发送信号终止进程
- 修复重复插入反向vma表的错误
- 添加test_filemap文件映射测试程序

* feat: 添加对内核引导协议的抽象 (#913)

* 添加multiboot header

* head.S传参增加bootloader类型

* feat: 添加引导加载协议的抽象,并为multiboot2实现这个抽象.

* 把framebuffer的映射地址改为从early ioremap和mmio pool分配

* riscv64能运行

* fix(virtio):修复了特定virtio设备环境下中断号重复错误，以及开机内核panic的bug (#881)

* feat: 允许通过multiboot引导（直到acpi初始化报错） (#914)

* fix(mm): 修复riscv64启动时的PageFault (#915)

* 修复riscv64启动时的PageFault

* 优化代码结构

* feat(driver/net): 将网络设备注册到sysfs, 添加设备类属性文件 (#919)

* ci(docs): Build docs when push/PR to master and release version. (#935)

Signed-off-by: longjin <[email protected]>

* fix: Fix make update-submodules-by-mirror Error (#928)

* remove --init suffix in update-submodules-by-mirror

* ci: deploy documents to minio (#936)

Signed-off-by: longjin <[email protected]>

* ci: fix awscli botcore (#937)

Signed-off-by: longjin <[email protected]>

* feat(ida): IDA内部改为使用XArray实现 (#934)

目前可以记录哪些ID已经分配,支持了ID释放的功能.

Signed-off-by: longjin <[email protected]>

* doc: add commit revision to footer while build with sphinx-multiversion (#939)

Signed-off-by: longjin <[email protected]>

* doc: 添加在github上编辑的按钮,以及扩展footer (#940)

Signed-off-by: longjin <[email protected]>

* fix: sysfs ifacetrait改名

* merge net && feat(net): stream sockpairs

---------

Signed-off-by: longjin <[email protected]>
Co-authored-by: MemoryShore <[email protected]>
Co-authored-by: 黄铭涛 <[email protected]>
Co-authored-by: LoGin <[email protected]>
Co-authored-by: linfeng <[email protected]>
Co-authored-by: Jomo <[email protected]>
Co-authored-by: Chiichen <[email protected]>
Co-authored-by: Samuel Dai <[email protected]>
Co-authored-by: Donkey Kane <[email protected]>
Co-authored-by: MemoryShore <[email protected]>
Co-authored-by: 曾俊 <[email protected]>
Co-authored-by: EMasi <[email protected]>

.github/workflows/docs-multiversion.yml

@@ -0,0 +1,47 @@
+name: Build multiversion docs
+
+on:
+  push:
+    branches: [ "master" ]
+    tags: [ "v*", "V*"]
+
+env:
+    GITHUB_REPOSITORY: ${{ github.repository }}
+    GITHUB_REF: ${{ github.ref }}
+
+jobs:
+
+  build-multiversion:
+    if: github.repository == 'DragonOS-Community/DragonOS'
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        fetch-depth: 0
+        ref: master
+
+    - name: install requirements
+      working-directory: ./docs
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y python3-pip python3-setuptools
+        pip3 install -r requirements.txt
+        python3 -m pip install --user awscli
+
+    - name: build docs
+      working-directory: ./docs
+      shell: bash -ileo pipefail {0}
+
+      run: |
+        make html-multiversion
+
+    - name: deploy docs
+      working-directory: ./docs
+      env:
+        AWS_ENDPOINT_URL: ${{ secrets.DOCS_DEPLOY_S3_ENDPOINT_URL }}
+        AWS_ACCESS_KEY_ID: ${{ secrets.DOCS_DEPLOY_S3_API_KEY }}
+        AWS_SECRET_ACCESS_KEY: ${{ secrets.DOCS_DEPLOY_S3_SECRET_KEY }}
+
+      run: |
+        aws s3 sync ./_build/html s3://dragonos-docs --delete

.github/workflows/docs.yml

@@ -0,0 +1,34 @@
+name: Build current version docs
+
+on:
+  pull_request:
+    branches: [ "master" ]
+    # paths:
+    #   - 'docs/**'
+
+env:
+    GITHUB_REPOSITORY: ${{ github.repository }}
+    GITHUB_REF: ${{ github.ref }}
+
+jobs:
+
+  build-current-version:
+    if: github.repository == 'DragonOS-Community/DragonOS'
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: install requirements
+      working-directory: ./docs
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y python3-pip
+        pip3 install -r requirements.txt
+
+    - name: build docs
+      working-directory: ./docs
+      shell: bash -ileo pipefail {0}
+
+      run: |
+        make html

.vscode/settings.json

@@ -129,7 +129,6 @@
         "assert.h": "c",
         "sys_version.h": "c",
         "cmd.h": "c",
-        "sleep.h": "c",
         "net.h": "c",
         "cmd_test.h": "c",
         "cmpxchg.h": "c",

docs/Makefile

@@ -8,6 +8,20 @@ SPHINXBUILD   ?= sphinx-build
 SOURCEDIR     = .
 BUILDDIR      = _build
 
+LANGUAGE      = zh_CN
+
+SPHINXOPTS	+= -D language=$(LANGUAGE)
+
+export CURRENT_GIT_COMMIT_HASH := $(shell git rev-parse --short HEAD)
+
+GIT_STATUS := $(shell git status --porcelain --untracked-files=no)
+
+ifeq ($(strip $(GIT_STATUS)),)
+    export CURRENT_GIT_COMMIT_DIRTY = 0
+else
+    export CURRENT_GIT_COMMIT_DIRTY = 1
+endif
+
 # Put it first so that "make" without argument is like "make help".
 help:
 	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
@@ -18,3 +32,7 @@ help:
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY:
+html-multiversion:
+	rm -rf ./$(BUILDDIR) && CURRENT_GIT_COMMIT_DIRTY=0 sphinx-multiversion $(SPHINXOPTS) "$(SOURCEDIR)" ./$(BUILDDIR)/html && cp -rf ./$(BUILDDIR)/html/master/* ./$(BUILDDIR)/html/

docs/_templates/footer.html

@@ -0,0 +1,8 @@
+{% extends "!footer.html" %}
+
+{% block extrafooter %}
+<div>
+    <br>
+    官网 <a href="https://dragonos.org">https://dragonos.org</a> | 论坛 <a href="https://bbs.dragonos.org.cn">https://bbs.dragonos.org.cn</a> |  <a href="https://dragonos.org">赞助DragonOS，成就未来！</a>
+</div>
+{% endblock %}

docs/_templates/versions.html

@@ -0,0 +1,27 @@
+{%- if current_version %}
+<div class="rst-versions" data-toggle="rst-versions" role="note" aria-label="versions">
+  <span class="rst-current-version" data-toggle="rst-current-version">
+    <span class="fa fa-book"> Other Versions</span>
+    v: {{ current_version.name }}
+    <span class="fa fa-caret-down"></span>
+  </span>
+  <div class="rst-other-versions">
+    {%- if versions.tags %}
+    <dl>
+      <dt>Tags</dt>
+      {%- for item in versions.tags %}
+      <dd><a href="{{ item.url }}">{{ item.name }}</a></dd>
+      {%- endfor %}
+    </dl>
+    {%- endif %}
+    {%- if versions.branches %}
+    <dl>
+      <dt>Branches</dt>
+      {%- for item in versions.branches %}
+      <dd><a href="{{ item.url }}">{{ item.name }}</a></dd>
+      {%- endfor %}
+    </dl>
+    {%- endif %}
+  </div>
+</div>
+{%- endif %}

docs/conf.py

@@ -19,6 +19,8 @@
 project = 'DragonOS'
 copyright = '2022-2024, DragonOS Community'
 author = 'longjin'
+github_org = 'DragonOS-Community'
+github_repo = 'DragonOS'
 
 # The full version, including alpha/beta/rc tags
 release = 'dev'
@@ -29,7 +31,7 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['myst_parser']
+extensions = ['myst_parser', 'sphinx_multiversion']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -54,11 +56,19 @@
 #
 html_theme = 'sphinx_rtd_theme'
 
+html_context = dict()
+
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
+html_sidebars = {
+    '**': [
+        'versioning.html',
+    ],
+}
+
 myst_enable_extensions = [
     "amsmath",
     "colon_fence",
@@ -74,10 +84,35 @@
     "tasklist",
 ]
 
+# sphinx-multiversion 指定哪个分支为 lastest 版本
+smv_latest_version = 'master'
+smv_released_pattern = r'^tags/.*$'           # Tags only
+smv_tag_whitelist = r'^(V.*|v.*)$'
+smv_branch_whitelist = "master"
 
 # Define the canonical URL if you are using a custom domain on Read the Docs
 html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "")
 
 # Tell Jinja2 templates the build is running on Read the Docs
 if os.environ.get("READTHEDOCS", "") == "True":
-    html_context["READTHEDOCS"] = True
+    html_context["READTHEDOCS"] = True
+
+if os.environ.get("SPHINX_MULTIVERSION_GIT_COMMIT", "") != "":
+    html_context["commit"] = os.environ["SPHINX_MULTIVERSION_GIT_COMMIT"]
+elif os.environ.get("CURRENT_GIT_COMMIT_HASH", "") != "":
+    html_context["commit"] = os.environ["CURRENT_GIT_COMMIT_HASH"]
+
+
+# 截取前 7 位 commit hash，如果长度不足则不截取
+if "commit" in html_context:
+    html_context["commit"] = html_context["commit"][:7]
+    if os.environ.get("CURRENT_GIT_COMMIT_DIRTY", "") == "1":
+        html_context["commit"] += "-dirty"
+
+
+# -- Set GitHub URL for Edit on GitHub links ---
+html_context['display_github'] = True
+html_context['github_user'] = github_org
+html_context['github_repo'] = github_repo
+html_context['github_version'] = html_context['commit'] if 'commit' in html_context else 'master'
+html_context['conf_py_path'] = '/docs/'

docs/kernel/boot/bootloader.md

@@ -1,12 +1,24 @@
 # 引导加载程序
 
-## 原理
+## X86_64
 
-  目前，DragonOS支持Legacy BIOS以及UEFI两种方式，进行启动引导。
+- [x] multiboot2
 
-  在`head.S`的头部包含了Multiboot2引导头，里面标志了一些Multiboot2相关的特定信息，以及一些配置命令。
+## RISC-V 64
 
-  在DragonOS的启动初期，会存储由GRUB2传来的magic number以及multiboot2_boot_info_addr。当系统进入`Start_Kernel`函数之后，将会把这两个信息保存到multiboot2驱动程序之中。信息的具体含义请参照Multiboot2 Specification进行理解，该部分难度不大，相信读者经过思考能理解其中的原理。
+DragonOS在RISC-V 64上，启动流程为：
+
+opensbi --> uboot --> DragonStub --> kernel
+
+这个启动流程，使得DragonOS内核与具体的硬件板卡解耦，能够以同一个二进制文件，在不同的硬件板卡上启动运行。
+
+
+## 内核启动回调
+
+DragonOS对内核引导加载程序进行了抽象，体现为`BootCallbacks`这个trait。
+不同的引导加载程序，实现对应的callback，初始化内核bootParams或者是其他的一些数据结构。
+
+内核启动时，自动根据引导加载程序的类型，注册回调。并且在适当的时候，会调用这些回调函数。
 
 ## 参考资料
 

docs/kernel/boot/index.rst

@@ -8,4 +8,3 @@
    :caption: 目录
 
    bootloader
-   multiboot2

docs/kernel/libs/id-allocation.md

@@ -0,0 +1,11 @@
+# ID分配
+
+:::{note}
+本文作者：龙进 <[email protected]>
+
+2024年9月25日
+:::
+
+内核提供了一个名为`IdAllocator`的ID分配器，位于`kernel/crates/ida`中。
+
+它能够分配、释放ID。默认它会自增分配，假如ID大于设定的最大值，它会从最小值开始寻找空闲ID。如果没有空闲的ID，则会分配失败。

docs/kernel/libs/index.rst

@@ -10,4 +10,5 @@
    lib_ui/scm
    lib_ui/textui
    unified-init
+   id-allocation
 

docs/requirements.txt

@@ -1,3 +1,4 @@
 sphinx==5.0.2
 myst-parser==0.18.0
-sphinx-rtd-theme
+sphinx-rtd-theme
+git+https://git.mirrors.dragonos.org.cn/DragonOS-Community/sphinx-multiversion.git@5858b75#egg=sphinx-multiversion

kernel/Cargo.toml

@@ -27,19 +27,19 @@ fatfs-secure = ["fatfs"]
 
 # 运行时依赖项
 [dependencies]
-acpi = { git = "https://git.mirrors.dragonos.org.cn/DragonOS-Community/acpi-rs.git", rev = "fb69243dcf" }
+acpi = { git = "https://git.mirrors.dragonos.org.cn/DragonOS-Community/acpi-rs.git", rev = "282df2af7b" }
 asm_macros = { path = "crates/asm_macros" }
 atomic_enum = "=0.2.0"
 bit_field = "=0.10"
 bitfield-struct = "=0.5.3"
 bitflags = "=1.3.2"
 bitmap = { path = "crates/bitmap" }
 driver_base_macros = { "path" = "crates/driver_base_macros" }
-# 一个no_std的hashmap、hashset
 elf = { version = "=0.7.2", default-features = false }
 fdt = { git = "https://git.mirrors.dragonos.org.cn/DragonOS-Community/fdt", rev = "9862813020" }
+# 一个no_std的hashmap、hashset
 hashbrown = "=0.13.2"
-ida = { path = "src/libs/ida" }
+ida = { path = "crates/ida" }
 intertrait = { path = "crates/intertrait" }
 kdepends = { path = "crates/kdepends" }
 klog_types = { path = "crates/klog_types" }
@@ -57,11 +57,13 @@ wait_queue_macros = { path = "crates/wait_queue_macros" }
 paste = "=1.0.14"
 slabmalloc = { path = "crates/rust-slabmalloc" }
 log = "0.4.21"
-
+lru = "0.12.3"
 
 # target为x86_64时，使用下面的依赖
 [target.'cfg(target_arch = "x86_64")'.dependencies]
 mini-backtrace = { git = "https://git.mirrors.dragonos.org.cn/DragonOS-Community/mini-backtrace.git", rev = "e0b1d90940" }
+multiboot = { path = "crates/multiboot" }
+multiboot2 = { git = "https://git.mirrors.dragonos.org.cn/DragonOS-Community/multiboot2", rev = "05739aab40" }
 raw-cpuid = "11.0.1"
 x86 = "=0.52.0"
 x86_64 = "=0.14.10"
@@ -88,4 +90,4 @@ debug = true   # Controls whether the compiler passes `-g`
 
 # The release profile, used for `cargo build --release`
 [profile.release]
-debug = false
+debug = true

kernel/src/libs/ida/Cargo.toml → kernel/crates/ida/Cargo.toml

@@ -3,7 +3,7 @@ name = "ida"
 version = "0.1.0"
 edition = "2021"
 authors = ["longjin <[email protected]>"]
-
-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+description = "一个基于XArray的ID分配器"
 
 [dependencies]
+kdepends = { path = "../kdepends" }

kernel/crates/ida/README.md

@@ -0,0 +1,3 @@
+# IDA
+
+一个基于XArray的ID分配器