YSOS-rust lab4

YSOS-lab4

1. 实验要求

了解用户态与内核态的区别、用户程序的加载与执行。
补充页表、内存分配相关知识，了解使用链接器脚本编译能够被加载执行的程序。
实现基本的系统调用。
实现用户程序的加载与执行，并切换到用户态。

2. 实验过程

milestone1

在一切配置顺利之后，应当可以使用 cargo build 在用户程序目录中正确地编译用户程序。

可以正常执行cargo build

milestone2

在 kernel/src/main.rs 初始化内核之后，尝试调用 list_app 函数，查看是否成功加载。

在完成了关键代码部分的[实现加载程序文件](# 实现加载程序文件)部分后，启动内核，效果如下图

内核启动后尝试调用proc::list_app();，可以看到成功加载了一个名为hello的程序

milestone3

踩坑记录

大致问题是，运行计算阶乘的测试程序时，如果输入的数很大，就需要分配一个很大的栈。

从输出看，在缺页异常的时候可以正常扩容栈空间，但是遇到了PROTECTION_VIOLATION的问题

dc0935ef5b9c3539a7dc656c94b6664

问题出在pkg/kernel/src/proc/process.rs

扩大栈空间需要复用之前实现的函数inc_stack_space，但它是为内核设计的，没有赋予用户访问的权限，将用户权限修改为true即可解决

milestone4

思考题

1.是否可以在内核线程中使用系统调用？并借此来实现同样的进程退出能力？分析并尝试回答。

可以在内核线程中使用系统调用，在kernel_main中加入syscall!(Syscall::Stat);，效果如下

在kernel_main启动shell之前加入syscall!(Syscall::Exit, 0);，结果内核进程真的退出了

内核进程完成kill_self之后，会调用manager.switch_next，切换下一个进程，但是就绪队列里没有进程，结果是内核进程只能继续执行，在访问页表的时候触发了panic

所以不能借此来实现内核进程的退出能力

2.为什么需要克隆内核页表？在系统调用的内核态下使用的是哪一张页表？用户态程序尝试访问内核空间会被正确拦截吗？尝试验证你的实现是否正确。

为了隔离用户空间和内核空间，避免用户访问内核空间内存。另外，还可以让所有用户进程使用同样的虚拟地址而不会相互干扰

在系统调用的内核态下使用的是内核的页表

修改用户程序hello，让它去读取内核栈的数据

let a:u8;
unsafe{
    //读取0xFFFFFF0100000000内存的数据
    a = *(0xFFFFFF0100000000 as *const u8);
}    
println!("a = {}", a);

运行结果如下

所以，用户态程序尝试访问内核空间会被正确拦截

3.为什么在使用 still_alive 函数判断进程是否存活时，需要关闭中断？在不关闭中断的情况下，会有什么问题？

这个坑我上次实验就踩过了，所以在still_alive函数调用的get_exit_code函数中关闭了中断，就不需要在still_alive函数关闭中断

这样做是为了避免死锁的产生

pkg/kernel/src/proc/manager.rs

impl ProcessManager {
	pub fn get_exit_code(&self, pid: &ProcessId) -> Option<i32> {
        x86_64::instructions::interrupts::without_interrupts(|| { 
            match self.get_proc(&pid){
                Some(proc) => {
                    match proc.read().exit_code(){
                        Some(code) => {
                            Some(code as i32)
                        },
                        None => {
                            None
                        }
                    }
                },
                None => None
            }
        })
    }
}

在执行stack的时候, 这个函数用于等待子进程退出, 当不使用without_interrupts时, 这段代码在第6行执行proc.read()会获得proc的读锁, 之后如果出现了时钟中断而读锁没有释放, 跳转到切换进程的代码.

在切换进程的过程中会调用pkg/kernel/src/proc/manager.rs的switch_next函数, 其中有一句代码

1	`proc.write().restore(context);`

需要获得proc写锁, 结果CPU会持续忙等, 等待读锁释放.

因为switch_next函数执行前已经关闭了中断, 这意味着新的时钟中断无法打断这个忙等, 死锁形成了

给get_exit_code添加without_interrupts之后, 就可以避免死锁发生

4.对于如下程序，使用 gcc 直接编译：

#include <stdio.h>

int main() {
    printf("Hello, World!\n");
    return 0;
}

从本次实验及先前实验的所学内容出发，结合进程的创建、链接、执行、退出的生命周期，参考系统调用的调用过程（可以仅以 Linux 为例），解释程序的运行。

进程创建：克隆一份内核的页表，注册一个新的进程加入就绪队列，然后将程序的elf文件加载到内存中，映射进程使用的栈
进程执行：操作系统的进程调度会选择该进程上处理机运行，运行的过程中需要调用printf库函数，库函数封装了对写操作的系统调用，用户程序只需将字符串放在内存中并传递地址和长度作为参数，由操作系统完成打印工作
进程退出：程序运行结束后会触发进程退出的系统调用，操作系统会销毁进程申请的资源，然后将处理机让给其他进程

5.x86_64::instructions::hlt 做了什么？为什么这样使用？为什么不可以在用户态中的 wait_pid 实现中使用？

x86_64::instructions::hlt用于实现内核的wait函数，它会被编译为HLT(halt)指令，这个指令的作用是将处理器置于空闲状态，直到下一个外部中断发生，这样可以降低CPU的功耗

HLT是一种特权指令，只能在内核态使用，所以不可以在用户态中的 wait_pid 实现中使用

6.有同学在某个回南天迷蒙的深夜遇到了奇怪的问题：

只有当进行用户输入（触发了串口输入中断）的时候，会触发奇怪的 Page Fault，然而进程切换、内存分配甚至 fork 等系统调用都很正常。

经过近三个小时的排查，发现他将 TSS 中的 privilege_stack_table 相关设置注释掉了。

请查阅资料，了解特权级栈的作用，实验说明这一系列中断的触发过程，尝试解释这个现象。

可以使用 intdbg 参数，或 ysos.py -i 进行数据捕获。
留意 0x0e 缺页异常和缺页之前的中断的信息。
注意到一个不应当存在的地址……？

或许你可以重新复习一下 Lab 2 的相关内容：double-fault-exceptions

注释掉 privilege_stack_table 之后，启动操作系统，按下键盘，观察到了 page fault 和 panic

如下图，这是由访问0xfffffffffffffff8导致的，由栈的范围可以看出，当前正在执行用户进程，但是访问量一个不在用户栈的进程，从而引发page fault，且无法处理

加分项123

1.😋尝试在 ProcessData 中记录代码段的占用情况，并统计当前进程所占用的页面数量，并在打印进程信息时，将进程的内存占用打印出来。

实现见关键代码部分，[点击跳转](# 加分项1：记录代码段占用情况)

只需要修改原有的print_process_list函数，增加统计内存的功能

2.😋 尝试在 kernel/src/memory/frames.rs 中实现帧分配器的回收功能 FrameDeallocator，作为一个最小化的实现，你可以在 Allocator 使用一个 Vec 存储被释放的页面，并在分配时从中取出。

实现见关键代码部分，[点击跳转](# 加分项2：FrameDeallocator)

用Vec<u32>数组存储回收的Frame，分配Frame时从数组中获取即可

3.🤔 基于帧回收器的实现，在 elf 中实现 unmap_range 函数，从页表中取消映射一段连续的页面，并使用帧回收器进行回收。之后，在合适的地方，结合 ProcessData 中存储的页面信息，利用这个函数实现进程栈的回收。其他进程资源（如页表、代码段、数据段等）的回收将会在后续实验中实现，目前暂时不需要考虑。

实现见关键代码部分，[点击跳转](# 加分项3：unmap_range)

实现 unmap_range 函数，然后在进程销毁时调用，就可以实现回收栈了

milestone5

在完成加分项123之前，如果连续运行三次fac计算999999的阶乘，会出现panic，原因是进程退出后栈没有回收，连续运行三次fac用完了物理内存

完成加分项123后，可以回收已退出进程的栈，避免panic的发生

加分项4

4.🤔 尝试利用 UefiRuntime 和 chrono crate，获取当前时间，并将其暴露给用户态，以实现 sleep 函数。

sleep实现见关键代码部分，[点击跳转](# 实现sleep)

给用户程序加入sleep函数

pkg/app/hello/src/main.rs

fn main() -> isize {
    println!("before sleep");
    let start = sys_time();
    println!("{:#?}",start);
    sleep(2000);
    let end = sys_time();
    println!("{:#?}", end);
    println!("Hello, world!!!");
}

两次打印之间确实停顿了2s

3. 关键代码

实现加载程序文件

修改boot配置

pkg/kernel/config/boot.conf

1	`load_apps=1`

==注意：== 最开始写load_apps=true, 结果无法正常加载程序, 经过排查,发现读取配置的底层逻辑(在pkg/boot/src/config.rs)为

let r10 = u64::from_str(value).unwrap_or(0);
match key {
    "load_apps" => self.load_apps = r10 != 0,
}

等号后面必须写数字来表示布尔值

修改boot代码

修改pkg/boot/src/lib.rs

定义了数据类型AppList和AppListRef, 结构体App, 并增加BootInfo的内容

use arrayvec::ArrayString;
use xmas_elf::ElfFile;
const MAX_APPLIST_LENGTH:usize = 16;
/// App information
pub struct App<'a> {
    /// The name of app
    pub name: ArrayString<MAX_APPLIST_LENGTH>,
    /// The ELF file
    pub elf: ElfFile<'a>,
}

pub type AppList = ArrayVec<App<'static>, MAX_APPLIST_LENGTH>;
//pub type AppListRef = Option<&'static[App<'static>]>;
pub type AppListRef = Option<&'static ArrayVec<App<'static>, MAX_APPLIST_LENGTH>>;
// pub type AppListRef<'a> = Option<&'a ArrayVec<App<'static>,MAX_APPLIST_LENGTH>>;
/// This structure represents the information that the bootloader passes to the kernel.
pub struct BootInfo {
    /// The memory map
    pub memory_map: MemoryMap,

    /// The offset into the virtual address space where the physical memory is mapped.
    pub physical_memory_offset: u64,

    /// UEFI SystemTable
    pub system_table: SystemTable<Runtime>,
    // Loaded apps
    pub loaded_apps: Option<AppList>,
}

修改pkg/boot/src/fs.rs, 增加load_apps函数

/// Load apps into memory, when no fs implemented in kernel
///
/// List all file under "APP" and load them.
pub fn load_apps(bs: &BootServices) -> AppList {
    let mut root = open_root(bs);
    let mut buf = [0; 8];
    let cstr_path = uefi::CStr16::from_str_with_buf("\\APP\\", &mut buf).unwrap();
    /* FIXME: get handle for \APP\ dir */
    let mut handle = root
    .open(cstr_path, FileMode::Read, FileAttribute::empty()).expect("Failed to open /APP")
    .into_directory().unwrap();

    let mut apps = ArrayVec::new();
    let mut entry_buf = [0u8; 0x100];

    loop {
        let info = handle
            .read_entry(&mut entry_buf)
            .expect("Failed to read entry");

        match info {
            Some(entry) => {
                /* FIXME: get handle for app binary file */
                let file = handle.open(entry.file_name(), FileMode::Read, FileAttribute::empty()).expect("Failed to open file");

                if file.is_directory().unwrap_or(true) {
                    continue;
                }

                let elf = {
                    // FIXME: load file with `load_file` function
                    let loaded_file = load_file(bs, &mut file.into_regular_file().unwrap());
                    // FIXME: convert file to `ElfFile`
                    ElfFile::new(loaded_file).expect("Failed to parse ELF file")
                };

                let mut name = ArrayString::<16>::new();
                entry.file_name().as_str_in_buf(&mut name).unwrap();

                apps.push(App { name, elf });
            }
            None => break,
        }
    }

    info!("Loaded {} apps", apps.len());

    apps
}

修改pkg/boot/src/main.rs

fn efi_main(image: uefi::Handle, mut system_table: SystemTable<Boot>) -> Status {
    // user apps
    let apps = if config.load_apps {
        info!("Loading apps...");
        Some(load_apps(system_table.boot_services()))
    } else {
        info!("Skip loading apps");
        None
    };
	//......
	// construct BootInfo
    let bootinfo = BootInfo {
        memory_map: mmap.entries().copied().collect(),
        physical_memory_offset: config.physical_memory_offset,
        system_table: runtime,
        loaded_apps: apps,
    };
}

修改kernel代码

修改pkg/boot/src/main.rs

pub fn kernel_main(boot_info: &'static boot::BootInfo) -> ! {
    ysos::init(boot_info);
    //ysos::wait(spawn_init());
    proc::list_app();
    ysos::shutdown(boot_info);
}

// pub fn spawn_init() -> proc::ProcessId {
//     // NOTE: you may want to clear the screen before starting the shell
//     // print_serial!("\x1b[1;1H\x1b[2J");

//     proc::list_app();
//     proc::spawn("sh").unwrap()
// }

修改pkg/kernel/src/proc/mod.rs

pub fn init(boot_info: &'static boot::BootInfo) {
    let app_list = boot_info.loaded_apps.as_ref();
    manager::init(kproc, app_list);
}

修改pkg/kernel/src/lib.rs

1
2
3

pub fn init(boot_info: &'static BootInfo) {
    proc::init(boot_info); // init process manager
}

修改pkg/kernel/src/proc/manager.rs

pub fn init(init: Arc<Process>, app_list:AppListRef) {

    // FIXME: set init process as Running
    init.write().resume();
    // FIXME: set processor's current pid to init's pid
    processor::set_pid(init.pid());
    PROCESS_MANAGER.call_once(|| ProcessManager::new(init, app_list));
}
pub struct ProcessManager {
    processes: RwLock<BTreeMap<ProcessId, Arc<Process>>>,
    ready_queue: Mutex<VecDeque<ProcessId>>,
    app_list: boot::AppListRef,
}
impl ProcessManager {
    pub fn app_list(&self) -> AppListRef {
        self.app_list
    }
    pub fn new(init: Arc<Process>, app_list:AppListRef) -> Self {
        let mut processes = BTreeMap::new();
        let ready_queue = VecDeque::new();
        let pid = init.pid();

        trace!("Init {:#?}", init);

        processes.insert(pid, init);
        Self {
            processes: RwLock::new(processes),
            ready_queue: Mutex::new(ready_queue),
            app_list: app_list,
        }
    }
}

==至此，可运行milestone2==

实现生成用户程序

将 ELF 文件从列表中取出，并生成用户程序：

pkg/kernel/src/proc/mod.rs

pub fn spawn(name: &str) -> Option<ProcessId> {
    let app = x86_64::instructions::interrupts::without_interrupts(|| {
        let app_list = get_process_manager().app_list()?;
        app_list.iter().find(|&app| app.name.eq(name))
    })?;

    elf_spawn(name.to_string(), &app.elf)
}

pub fn elf_spawn(name: String, elf: &ElfFile) -> Option<ProcessId> {
    let pid = x86_64::instructions::interrupts::without_interrupts(|| {
        let manager = get_process_manager();
        let process_name = name.to_lowercase();
        let parent = Arc::downgrade(&manager.current());
        let pid = manager.spawn(elf, name, Some(parent), None);

        debug!("Spawned process: {}#{}", process_name, pid);
        pid
    });

    Some(pid)
}

在 ProcessManager 中，实现 spawn 函数：

pkg/kernel/src/proc/manager.rs

impl ProcessManager {
	pub fn spawn(
        &self,
        elf: &ElfFile,
        name: String,
        parent: Option<Weak<Process>>,
        proc_data: Option<ProcessData>,
    ) -> ProcessId {
        let kproc = self.get_proc(&KERNEL_PID).unwrap();
        let page_table = kproc.read().clone_page_table();
        let page_table_mapper: x86_64::structures::paging::OffsetPageTable<'static> = page_table.mapper();
        let proc = Process::new(name, parent, page_table, proc_data);
        let pid = proc.pid();

        let mut inner = proc.write();
        // FIXME: load elf to process pagetable
        inner.load_elf(elf, page_table_mapper);
        // FIXME: alloc new stack for process
        
        inner.set_stack_frame(VirtAddr::new_truncate(elf.header.pt2.entry_point()), VirtAddr::new_truncate(STACK_INIT_TOP));
        // FIXME: mark process as ready
        inner.pause();
        drop(inner);

        trace!("New {:#?}", &proc);

        // FIXME: something like kernel thread
        self.add_proc(pid, proc);
        self.push_ready(pid);
        pid
    }
}

注释掉上个实验的spawn_kernel_thread，防止后续修改后的进程模型在执行内核线程时遇到意外的问题。

pkg/kernel/src/proc/process.rs

impl ProcessInner {
	pub fn load_elf(&mut self, elf:&ElfFile, mut mapper: x86_64::structures::paging::OffsetPageTable<'static>){
        let alloc = &mut *get_frame_alloc_for_sure();
        elf::load_elf(
            elf, 
            *PHYSICAL_OFFSET.get().unwrap(), 
            &mut mapper, 
            alloc, 
            true
        ).unwrap();
        let stack_segment = elf::map_range(STACK_INIT_BOT, STACK_DEF_PAGE, &mut mapper, alloc, true, false).unwrap();
        //info!("stack segment: {:?}", stack_segment);
        self.proc_data.as_mut().unwrap().set_stack(VirtAddr::new(STACK_INIT_TOP), STACK_DEF_PAGE);
        self.proc_data.as_mut().unwrap().set_max_stack(VirtAddr::new(STACK_MAX - STACK_MAX_SIZE), STACK_MAX_PAGES);
    }
}

修改`pkg/elf/src/lib.rs`

pub fn map_range(
    addr: u64,
    count: u64,
    page_table: &mut impl Mapper<Size4KiB>,
    frame_allocator: &mut impl FrameAllocator<Size4KiB>,
    user_access: bool,
    no_execute: bool,
)
-> Result<PageRange, MapToError<Size4KiB>> {
    let mut flags = PageTableFlags::PRESENT | PageTableFlags::WRITABLE;

    if user_access {
        flags |= PageTableFlags::USER_ACCESSIBLE;
    }
    if no_execute{
        flags |= PageTableFlags::NO_EXECUTE;
    }
}
pub fn load_elf(
    elf: &ElfFile,
    physical_offset: u64,
    page_table: &mut impl Mapper<Size4KiB>,
    frame_allocator: &mut impl FrameAllocator<Size4KiB>,
    user_access: bool,
) -> Result<(), MapToError<Size4KiB>> {
    for segment in elf.program_iter() {
        if segment.get_type().unwrap() != program::Type::Load {
            continue;
        }

        load_segment(
            file_buf,
            physical_offset,
            &segment,
            page_table,
            frame_allocator,
            user_access,
        )?
    }

    Ok(())
}
fn load_segment(
    file_buf: *const u8,
    physical_offset: u64,
    segment: &program::ProgramHeader,
    page_table: &mut impl Mapper<Size4KiB>,
    frame_allocator: &mut impl FrameAllocator<Size4KiB>,
    user_access: bool,
) -> Result<(), MapToError<Size4KiB>> {
    if user_access {
        page_table_flags |= PageTableFlags::USER_ACCESSIBLE;
    }
}

修改GDT

pkg/kernel/src/proc/context.rs

impl ProcessContext {
	pub fn init_stack_frame(&mut self, entry: VirtAddr, stack_top: VirtAddr) {
        self.value.stack_frame.stack_pointer = stack_top;
        self.value.stack_frame.instruction_pointer = entry;
        self.value.stack_frame.cpu_flags =
            RFlags::IOPL_HIGH | RFlags::IOPL_LOW | RFlags::INTERRUPT_FLAG;

        let selector = get_selector();
        self.value.stack_frame.code_segment = selector.code_selector;
        self.value.stack_frame.stack_segment = selector.data_selector;

        trace!("Init stack frame: {:#?}", &self.stack_frame);
        let selector = get_user_selector(); // FIXME: implement this function

        self.value.stack_frame.code_segment = selector.user_code_selector;
        self.value.stack_frame.stack_segment = selector.user_data_selector;
    }
}

pkg/kernel/src/memory/gdt.rs

pub const SYSCALL_IST_INDEX: u16 = 3;
pub const IST_SIZES: [usize; 5] = [0x1000, 0x1000, 0x1000, 0x1000, 0x1000];

lazy_static! {
    static ref TSS: TaskStateSegment = {
        tss.interrupt_stack_table[SYSCALL_IST_INDEX as usize] = {
            const STACK_SIZE: usize = IST_SIZES[4];
            static mut STACK: [u8; STACK_SIZE] = [0; STACK_SIZE];
            let stack_start = VirtAddr::from_ptr(unsafe { STACK.as_ptr() });
            let stack_end = stack_start + STACK_SIZE as u64;
            info!(
                "SYSCALL Stack  : 0x{:016x}-0x{:016x}",
                stack_start.as_u64(),
                stack_end.as_u64()
            );
            stack_end
        };
        // You can use `tss.interrupt_stack_table[DOUBLE_FAULT_IST_INDEX as usize]`

        tss
    };
}

lazy_static! {
    static ref GDT: (GlobalDescriptorTable, KernelSelectors, UserSelectors) = {
        let mut gdt = GlobalDescriptorTable::new();
        let code_selector = gdt.append(Descriptor::kernel_code_segment());
        let data_selector = gdt.append(Descriptor::kernel_data_segment());
        let tss_selector = gdt.append(Descriptor::tss_segment(&TSS));
        let user_code_selector = gdt.append(Descriptor::user_code_segment());
        let user_data_selector = gdt.append(Descriptor::user_data_segment());
        (
            gdt,
            KernelSelectors {
                code_selector,
                data_selector,
                tss_selector,
            },
            UserSelectors {
                user_code_selector,
                user_data_selector,
            },
        )
    };
}
#[derive(Debug)]
pub struct UserSelectors {
    pub user_code_selector: SegmentSelector,
    pub user_data_selector: SegmentSelector,
}
pub fn get_user_selector() -> &'static UserSelectors {
    &GDT.2
}

实现系统调用

补全中断注册函数和`dispatcher`

pkg/kernel/src/interrupt/mod.rs

pub mod syscall;
lazy_static! {
    static ref IDT: InterruptDescriptorTable = {
        let mut idt = InterruptDescriptorTable::new();
        unsafe {
            exceptions::register_idt(&mut idt);
            clock::register_idt(&mut idt);
            serial::register_idt(&mut idt);
            syscall::register_idt(&mut idt);    
        }
        idt
    };
}

==注意==：不要漏了syscall::register_idt(&mut idt);(困扰CJL一个下午)

pkg/kernel/src/interrupt/syscall/mod.rs

pub unsafe fn register_idt(idt: &mut InterruptDescriptorTable) {
    // FIXME: register syscall handler to IDT
    //        - standalone syscall stack
    //        - ring 3
    idt[consts::Interrupts::Syscall as u8]
        .set_handler_fn(syscall_handler)
        .set_stack_index(SYSCALL_IST_INDEX)
        .set_privilege_level(x86_64::PrivilegeLevel::Ring3);
}
pub fn dispatcher(context: &mut ProcessContext) {
    let args = super::syscall::SyscallArgs::new(
        Syscall::from(context.regs.rax),
        context.regs.rdi,
        context.regs.rsi,
        context.regs.rdx,
    );

    // NOTE: you may want to trace syscall arguments
    //trace!("{}", args);

    match args.syscall {
        // fd: arg0 as u8, buf: &[u8] (ptr: arg1 as *const u8, len: arg2)
        Syscall::Read => { /* FIXME: read from fd & return length */
            context.set_rax(sys_read(&args))
        },
        // fd: arg0 as u8, buf: &[u8] (ptr: arg1 as *const u8, len: arg2)
        Syscall::Write => { /* FIXME: write to fd & return length */
            context.set_rax(sys_write(&args))
        },

        // None -> pid: u16
        Syscall::GetPid => { /* FIXME: get current pid */ 
            context.set_rax(get_pid() as usize)
        },

        // path: &str (ptr: arg0 as *const u8, len: arg1) -> pid: u16
        Syscall::Spawn => { /* FIXME: spawn process from name */
            context.set_rax(spawn_process(&args))
        },
        // ret: arg0 as isize
        Syscall::Exit => { /* FIXME: exit process with retcode */
            exit_process(&args, context)
        },
        // pid: arg0 as u16 -> status: isize
        Syscall::WaitPid => { /* FIXME: check if the process is running or get retcode */
            context.set_rax(wait_pid(&args))
        },

        // None
        Syscall::Stat => { /* FIXME: list processes */ 
            list_process()
        },
        // None
        Syscall::ListApp => { /* FIXME: list available apps */
            service_list_app()
        },

        // ----------------------------------------------------
        // NOTE: following syscall examples are implemented
        // ----------------------------------------------------

        // layout: arg0 as *const Layout -> ptr: *mut u8
        Syscall::Allocate => context.set_rax(sys_allocate(&args)),
        // ptr: arg0 as *mut u8
        Syscall::Deallocate => sys_deallocate(&args),
        // Unknown
        Syscall::Unknown => warn!("Unhandled syscall: {:x?}", context.regs.rax),
    }
}

实现中断处理函数

pkg/kernel/src/interrupt/syscall/service.rs

pub fn sys_write(args: &SyscallArgs) -> usize {//arg0区分标准输出和错误输出，arg1为指针，arg2为字符串长度
    // FIXME: get buffer and fd by args
    //       - core::slice::from_raw_parts
    //info!("at sys_write");
    let buf: &[u8];
    unsafe{
        buf = core::slice::from_raw_parts(args.arg1 as *const u8, args.arg2 as usize);
    }
    // FIXME: call proc::write -> isize
    //let res = proc::write(args.syscall.clone() as u8, buf);
    let res = proc::write(args.arg0 as u8, buf);
    // FIXME: return the result as usize
    res as usize
}

pub fn sys_read(args: &SyscallArgs) -> usize {
    // FIXME: just like sys_write
    let buf;
    unsafe{
        buf = core::slice::from_raw_parts_mut(args.arg1 as *mut u8, args.arg2 as usize);
    };
    //let res = proc::read(args.syscall.clone() as u8, buf);
    let res = proc::read(args.arg0 as u8, buf);
    res as usize
}

pub fn exit_process(args: &SyscallArgs, context: &mut ProcessContext) {
    // FIXME: exit process with retcode
    proc::exit(args.arg0 as isize, context)
}

pub fn list_process() {
    // FIXME: list all processes
    proc::print_process_list();
}
pub fn get_pid() -> u16{
    get_process_manager().current().pid().0
}
pub fn service_list_app(){
    proc::list_app();
}
pub fn wait_pid(args: &SyscallArgs){
    get_process_manager().get_exit_code(&ProcessId(args.arg0 as u16));
}

实现动态内存分配

pkg/kernel/src/memory/user.rs

1
2
3

pub fn init_user_heap() -> Result<(), MapToError<Size4KiB>> {
    elf::map_range(USER_HEAP_START as u64, USER_HEAP_PAGE as u64, mapper, frame_allocator, true, true).unwrap();
}

pkg/kernel/src/memory/mod.rs

1
2
3

pub fn init(boot_info: &'static boot::BootInfo) {
	user::init();
}

实现标准输入输出

实现用户态库read_line

pkg/lib/src/io.rs

impl Stdin {
	pub fn read_line(&self) -> String {
        // FIXME: allocate string
        let mut s = String::new();
        // FIXME: read from input buffer
        let mut buf = vec![0;4];
        //s = sys_read(1, buf);
        //       - maybe char by char?
        loop{
            match sys_read(0, &mut buf){
                Some(n)=>{
                    if n > 0{
                        let ch = String::from_utf8_lossy(&buf).to_string().remove(0);
                        match ch{
                            '\n'|'\r' => {
                                self::print!("\n");
                                break;
                            },
                            '\x03' =>{//ctrl-C
                                s.clear();
                                self::print!("^C\n");
                                break;
                            },
                            '\x08' | '\x7f' => {
                                if !s.is_empty() {
                                    self::print!("\x08\x20\x08");
                                    s.pop();
                                }
                            },
                            _ => {
                                s.push(ch);
                            }
                        }
                    }
                },
                None=>{
                    continue;
                }
            }
        }
        // FIXME: handle backspace / enter...
        // FIXME: return string
        s
    }
}

内核态处理输入输出

pkg/kernel/src/proc/mod.rs

pub fn read(fd: u8, buf: &mut [u8]) -> isize {
    x86_64::instructions::interrupts::without_interrupts(|| get_process_manager().current().read().read(fd, buf))
}

pub fn write(fd: u8, buf: &[u8]) -> isize {
    x86_64::instructions::interrupts::without_interrupts(|| get_process_manager().current().read().write(fd, buf))
}

pkg/kernel/src/proc/data.rs

impl ProcessData {
    pub fn read(&self, fd: u8, buf: &mut [u8]) -> isize {
        self.resources.read().read(fd, buf)
    }

    pub fn write(&self, fd: u8, buf: &[u8]) -> isize {
        self.resources.write().write(fd, buf)
    }
}

修改 ProcessData 结构体，类似于环境变量的定义，添加一个“文件描述符表”：

#[derive(Debug, Clone)]
pub struct ProcessData {
    // shared data
    pub(super) env: Arc<RwLock<BTreeMap<String, String>>>,

    // process specific data
    pub(super) stack_segment: Option<PageRange>,
    pub(super) max_stack_segment: Option<PageRange>,
    pub(super) resources: Arc<RwLock<ResourceSet>>
}

impl Default for ProcessData {
    fn default() -> Self {
        Self {
            env: Arc::new(RwLock::new(BTreeMap::new())),
            stack_segment: None,
            max_stack_segment: None,
            resources: Arc::new(RwLock::new(ResourceSet::default())),
        }
    }
}

pkg/kernel/src/utils/resource.rs

impl Resource {
    pub fn read(&mut self, buf: &mut [u8]) -> Option<usize> {
        match self {
            Resource::Console(stdio) => match stdio {
                StdIO::Stdin => {
                    // FIXME: just read from kernel input buffer
                    if buf.len() < 4{
                        error!("buf length should >= 4");
                        Some(0)
                    }
                    else{
                        let k = try_pop_key();
                        match k{
                            None => Some(0),
                            Some(k) => {
                                match k{
                                    DecodedKey::RawKey(r) =>{
                                        warn!("Rawkey {:#?} is not supported",r);
                                        Some(0)
                                    }
                                    DecodedKey::Unicode(c) => {
                                        Some(c.encode_utf8(&mut buf[0..4]).len())
                                    }
                                }
                            },
                        }
                    }
                    
                }
                _ => None,
            },
            Resource::Null => Some(0),
        }
    }
}

实现进程退出

pkg/kernel/src/proc/mod.rs

pub fn exit(ret: isize, context: &mut ProcessContext) {
    x86_64::instructions::interrupts::without_interrupts(|| {
        let manager = get_process_manager();
        // FIXME: implement this for ProcessManager
        manager.kill_self(ret);
        manager.switch_next(context);
    })
}

pkg/kernel/src/proc/manager.rs

1
2
3

pub fn kill_self(&self, ret: isize) {
    self.kill(processor::get_pid(), ret);
}

pkg/lib/src/macros.rs

#[macro_export]
macro_rules! entry {
    ($fn:ident) => {
        #[export_name = "_start"]
        pub extern "C" fn __impl_start() {
            let ret = $fn();
            // FIXME: after syscall, add lib::sys_exit(ret);
            lib::sys_exit(ret);
        }
    };
}

#[cfg_attr(not(test), panic_handler)]
fn panic(info: &core::panic::PanicInfo) -> ! {
    let location = if let Some(location) = info.location() {
        alloc::format!(
            "{}:{}:{}",
            location.file(),
            location.line(),
            location.column()
        )
    } else {
        "Unknown location".to_string()
    };
    let msg = if let Some(msg) = info.message() {
        alloc::format!("{}", msg)
    } else {
        "No more message...".to_string()
    };
    errln!("\n\n\rERROR: panicked at {}\n\n\r{}", location, msg);

    // FIXME: after syscall, add lib::sys_exit(1);
    super::sys_exit(1);
    unreachable!()
}

实现`main`函数

pkg/app/hello/src/main.rs

#![no_std]
#![no_main]

use lib::*;

extern crate lib;

fn main() -> isize {
    println!("Hello, world!!!");
    233
}

entry!(main);

pkg/app/hello/src/main.rs

#![no_std]
#![no_main]

use log::debug;
use ysos::*;
use ysos_kernel as ysos;

extern crate alloc;

boot::entry_point!(kernel_main);

pub fn kernel_main(boot_info: &'static boot::BootInfo) -> ! {
    ysos::init(boot_info);
    spawn_init();
    //loop{}
    loop {
        print!("[>] ");
        let line = input::get_line();
        match line.trim() {
            "exit" => break,
            "ps" => {
                //print!("\n");
                ysos::proc::print_process_list();
            }
            _ => println!("[=] {}", line),
        }
    }
    ysos::shutdown(boot_info);
}

pub fn spawn_init() -> proc::ProcessId {
    // NOTE: you may want to clear the screen before starting the shell
    // print_serial!("\x1b[1;1H\x1b[2J");

    proc::list_app();
    proc::spawn("hello").unwrap()
}

==至此，可运行milestone3==

在内核实现进程的创建和等待

这样，在内核调用wait，可以实现等待shell进程退出

pkg/kernel/src/lib.rs

pub fn wait(init: proc::ProcessId) {
    loop {
        if proc::still_alive(init) {
            // Why? Check reflection question 5
            x86_64::instructions::hlt();
        } else {
            break;
        }
    }
}

pkg/kernel/src/proc/mod.rs

#[inline]
pub fn still_alive(pid: ProcessId) -> bool {
    //x86_64::instructions::interrupts::without_interrupts(|| {
        // check if the process is still alive
        let pid = get_process_manager().get_exit_code(&pid);
        if let None = pid {
            true
        }else {
            false
        }
    //})
}

实现用户态的进程创建和等待

用户态库的进程等待

pkg/lib/src/syscall.rs

#[inline(always)]
pub fn sys_wait_pid(pid: u16) -> isize {
    // FIXME: try to get the return value for process
    //        loop until the process is finished
    loop {
        let status = syscall!(Syscall::WaitPid, pid as u64) as isize;
        if status != 114514 {
            // Why? Check reflection question 5
            //x86_64::instructions::hlt();
            break;
        }
    }
    0
}

系统调用支持进程创建和等待

pkg/kernel/src/interrupt/syscall/service.rs

pub fn spawn_process(args: &SyscallArgs) -> usize {
    // FIXME: get app name by args
    //       - core::str::from_utf8_unchecked
    //       - core::slice::from_raw_parts
    // FIXME: spawn the process by name
    // FIXME: handle spawn error, return 0 if failed
    // FIXME: return pid as usize
    let buf: &[u8];
    unsafe{
        buf = core::slice::from_raw_parts(args.arg0 as *const u8, args.arg1 as usize);
    }
    let name;
    unsafe {
        name = core::str::from_utf8_unchecked(buf);
    };
    //debug!("name is {}",name);
    let res = proc::spawn(name);
    match res {
        Some(pid) => pid.0 as usize,
        None => 0,
    }
}
pub fn wait_pid(args: &SyscallArgs)->usize{
    let res = get_process_manager().get_exit_code(&ProcessId(args.arg0 as u16));
    match res {
        Some(code) => code as usize,
        None => 114514,
    }
}

实现main函数

内核

pkg/kernel/src/main.rs

#![no_std]
#![no_main]

use log::debug;
use ysos::*;
use ysos_kernel as ysos;

extern crate alloc;

boot::entry_point!(kernel_main);

pub fn kernel_main(boot_info: &'static boot::BootInfo) -> ! {
    ysos::init(boot_info);
    wait(spawn_init());
    ysos::shutdown(boot_info);
}

pub fn spawn_init() -> proc::ProcessId {
    proc::spawn("shell").unwrap()
}

shell

pkg/app/shell/src/main.rs

#![no_std]
#![no_main]

use lib::*;

extern crate lib;

fn main() -> isize {
    loop {
        print!("[>] ");
        let line = stdin().read_line();
        match line.trim() {
            "exit" => break,
            "app" => sys_list_app(),
            "ps" => sys_stat(),
            "hello" => {sys_wait_pid(sys_spawn("hello"));},
            "fac" => {sys_wait_pid(sys_spawn("fac"));},
            "clear" => {print!("\x1b[2J\x1b[1;1H");},
            "help" => {print_help();},
            _ => println!("[=] {}", line),
        }
    }
    0
}

entry!(main);
fn print_help(){
    println!("\x1b[34;1;4m[*] help:\x1b[0m
    \x1b[34mYSOS shell 使用帮助\x1b[0m
    author: CJL-22330004
    \x1b[32m指令:
    - exit: 退出
    - ps: 展示当前所有进程
    - app: 展示所有用户程序
    - hello: 运行用户程序hello
    - fac: 运行用户程序fac, 用于计算阶乘
    - clear: 清屏
    - help: 打印帮助信息\x1b[0m");
}

hello

pkg/app/hello/src/main.rs

#![no_std]
#![no_main]

use lib::*;

extern crate lib;

fn main() -> isize {
    println!("Hello, world!!!");
    print!("Please input a line:");
    let stdin1 = stdin();
    let s = stdin1.read_line();
    println!("{}", s);
    
    //loop{}
    233
}

entry!(main);

fac

pkg/app/fac/src/main.rs

#![no_std]
#![no_main]

use lib::*;

extern crate lib;
const MOD: u64 = 1000000007;

fn factorial(n: u64) -> u64 {
    if n == 0 {
        1
    } else {
        n * factorial(n - 1) % MOD
    }
}

fn main() -> isize {
    print!("Input n: ");

    let input = lib::stdin().read_line();

    // prase input as u64
    let n = input.parse::<u64>().unwrap();

    if n > 1000000 {
        println!("n must be less than 1000000");
        return 1;
    }

    // calculate factorial
    let result = factorial(n);

    // print system status
    sys_stat();

    // print result
    println!("The factorial of {} under modulo {} is {}.", n, MOD, result);

    0
}

entry!(main);

实现sleep

实现UefiRuntime

pkg/kernel/src/interrupt/clock.rs

once_mutex!(UEFI_SERVICE: UefiRuntime);
pub fn init(boot_info: &'static BootInfo) {
    unsafe {
        init_UEFI_SERVICE(UefiRuntime::new(boot_info));
    }
}
guard_access_fn! {
    pub get_uefi_runtime(UEFI_SERVICE: UefiRuntime)
}
pub struct UefiRuntime {
    runtime_service: &'static RuntimeServices,
}

impl UefiRuntime {
    pub unsafe fn new(boot_info: &'static BootInfo) -> Self {
        Self {
            runtime_service: boot_info.system_table.runtime_services(),
        }
    }

    pub fn get_time(&self) -> Time {
        self.runtime_service.get_time().unwrap()
    }
}
pub fn now() -> NaiveDateTime {
    let time = get_uefi_runtime_for_sure().get_time();
    NaiveDate::from_ymd_opt(time.year() as i32, time.month() as u32, time.day() as u32)
        .unwrap_or_default()
        .and_hms_nano_opt(
            time.hour() as u32,
            time.minute() as u32,
            time.second() as u32,
            time.nanosecond(),
        )
        .unwrap_or_default()
}

pkg/kernel/src/lib.rs

1
2
3

pub fn init(boot_info: &'static BootInfo) {
    clock::init(boot_info);
}

实现系统调用

pkg/kernel/src/interrupt/syscall/mod.rs

1
2
3

Syscall::Time => {
            context.set_rax(sys_clock() as usize)
        },

pkg/kernel/src/interrupt/syscall/service.rs

1
2
3

pub fn sys_clock() -> i64{
    clock::now().timestamp_nanos_opt().unwrap_or_default()
}

pkg/lib/src/lib.rs

pub fn sleep(millisecs: i64) {
    let start = sys_time();
    let dur = Duration::milliseconds(millisecs);
    let mut current = start;
    while current - start < dur {
        current = sys_time();
    }
}

pkg/lib/src/syscall.rs

#[inline(always)]
pub fn sys_time() -> NaiveDateTime {
    let time = syscall!(Syscall::Time) as i64;
    const BILLION: i64 = 1_000_000_000;
    NaiveDateTime::from_timestamp_opt(time / BILLION, (time % BILLION) as u32).unwrap_or_default()
}

加分项1：记录代码段占用情况

pkg/kernel/src/proc/manager.rs

pub fn print_process_list(&self) {
    let mut output = String::from("  PID | PPID | Process Name |  Ticks  | Status\n");

    self.processes
    .read()
    .values()
    .filter(|p| p.read().status() != ProgramStatus::Dead)
    .for_each(|p| output += format!("{}\n", p).as_str());

    // TODO: print memory usage of kernel heap
    let heap_used = ALLOCATOR.lock().used();
    let heap_size = HEAP_SIZE;

    let user_heap_used = USER_ALLOCATOR.lock().used();
    let user_heap_size = USER_HEAP_SIZE;

    let alloc = get_frame_alloc_for_sure();
    let frames_used = alloc.frames_used();
    let frames_recycled = alloc.recycled_count();
    let frames_total = alloc.frames_total();

    let (sys_used, sys_used_unit) = crate::humanized_size(heap_used as u64);
    let (sys_size, sys_size_unit) = crate::humanized_size(heap_size as u64);

    output += format!(
        "Kernel : {:>6.*} {} / {:>6.*} {} ({:>5.2}%)\n",
        2,
        sys_used,
        sys_used_unit,
        2,
        sys_size,
        sys_size_unit,
        heap_used as f64 / heap_size as f64 * 100.0
    )
    .as_str();

    let (user_used, user_used_unit) = crate::humanized_size(user_heap_used as u64);
    let (user_size, user_size_unit) = crate::humanized_size(user_heap_size as u64);

    output += format!(
        "User   : {:>6.*} {} / {:>6.*} {} ({:>5.2}%)\n",
        2,
        user_used,
        user_used_unit,
        2,
        user_size,
        user_size_unit,
        user_heap_used as f64 / user_heap_size as f64 * 100.0
    )
    .as_str();

    // put used/total frames in MiB
    let (used_size, used_unit) =
    crate::humanized_size((frames_used - frames_recycled) as u64 * PAGE_SIZE);
    let (tot_size, tot_unit) = crate::humanized_size(frames_total as u64 * PAGE_SIZE);

    output += format!(
        "Memory : {:>6.*} {} / {:>6.*} {} ({:>5.2}%) [{} recycled]\n",
        2,
        used_size,
        used_unit,
        2,
        tot_size,
        tot_unit,
        (frames_used - frames_recycled) as f64 / frames_total as f64 * 100.0,
        frames_recycled
    )
    .as_str();

    output += format!("Queue  : {:?}\n", self.ready_queue.lock()).as_str();

    output += &processor::print_processors();

    print!("{}", output);
}

加分项2：FrameDeallocator

pkg/kernel/src/memory/frames.rs

pub struct BootInfoFrameAllocator {
    size: usize,
    used: usize,
    frames: BootInfoFrameIter,
    recycled: Vec<u32>,
}
impl BootInfoFrameAllocator {
    /// Create a FrameAllocator from the passed memory map.
    ///
    /// This function is unsafe because the caller must guarantee that the passed
    /// memory map is valid. The main requirement is that all frames that are marked
    /// as `USABLE` in it are really unused.
    pub unsafe fn init(memory_map: &MemoryMap, size: usize) -> Self {
        BootInfoFrameAllocator {
            size,
            frames: create_frame_iter(memory_map),
            used: 0,
            recycled: Vec::new(),
        }
    }
    pub fn recycled_count(&self) -> usize {
        self.recycled.len() as usize
    }
}
unsafe impl FrameAllocator<Size4KiB> for BootInfoFrameAllocator {
    fn allocate_frame(&mut self) -> Option<PhysFrame> {
        if let Some(frame) = self.recycled.pop() {
            Some(u32_to_phys_frame(frame))
        } else {
            self.used += 1;
            self.frames.next()
        }
    }
}
impl FrameDeallocator<Size4KiB> for BootInfoFrameAllocator {
    unsafe fn deallocate_frame(&mut self, frame: PhysFrame) {
        // TODO: deallocate frame (not for lab 2)
        let key = phys_frame_to_u32(frame);
        self.recycled.push(key);
    }
}

const RS_ALIGN_4KIB: u64 = 12;

/// Assumes that the physical memory we have is less than 16TB
/// and we can store the 4KiB aligned address in a u32
#[inline(always)]
fn phys_frame_to_u32(frame: PhysFrame) -> u32 {
    let key = frame.start_address().as_u64() >> RS_ALIGN_4KIB;

    assert!(key <= u32::MAX as u64);

    key as u32
}

#[inline(always)]
fn u32_to_phys_frame(key: u32) -> PhysFrame {
    PhysFrame::containing_address(PhysAddr::new((key as u64) << RS_ALIGN_4KIB))
}

加分项3：unmap_range

pkg/elf/src/lib.rs

pub fn unmap_range(
    addr: u64,
    pages: u64,
    page_table: &mut impl Mapper<Size4KiB>,
    frame_deallocator: &mut impl FrameDeallocator<Size4KiB>,
    do_dealloc: bool,
) -> Result<(), UnmapError> {
    trace!("Unmapping stack at {:#x}", addr);

    let range_start = Page::containing_address(VirtAddr::new(addr));

    trace!(
        "Mem range hint: {:#x} -> {:#x}",
        addr,
        page_table
            .translate_page(range_start)
            .unwrap()
            .start_address()
    );

    let range_end = range_start + pages;

    trace!(
        "Page Range: {:?}({})",
        Page::range(range_start, range_end),
        pages
    );

    for page in Page::range(range_start, range_end) {
        let info = page_table.unmap(page)?;
        if do_dealloc {
            unsafe {
                frame_deallocator.deallocate_frame(info.0);
            }
        }
        info.1.flush();
    }

    Ok(())
}

然后修改进程退出的代码，调用clean_up_stack函数销毁栈

pkg/kernel/src/proc/process.rs

fn clean_up_stack(&mut self, pid: ProcessId) {
        let page_table = self.page_table.take().unwrap();
        let mut mapper = page_table.mapper();

        let frame_deallocator = &mut *get_frame_alloc_for_sure();
        let start_count = frame_deallocator.recycled_count();

        let proc_data = self.proc_data.as_mut().unwrap();
        let stack = proc_data.stack_segment.unwrap();

        debug!(
            "Free stack for {}#{}: [{:#x} -> {:#x}) ({} frames)",
            self.name,
            pid,
            stack.start.start_address(),
            stack.end.start_address(),
            stack.count()
        );

        elf::unmap_range(
            stack.start.start_address().as_u64(),
            stack.count() as u64,
            &mut mapper,
            frame_deallocator,
            true,
        )
        .unwrap();

    }

4. 实验结果

见milestone4，[点击跳转](# milestone4)

5. 总结

了解了操作系统上如何运行用户程序和系统调用的实现

courses > YSOS

YSOS-rust lab4

https://blog.algorithmpark.xyz/2024/05/19/YSOS/lab4/index/

作者

CJL

发布于

2024年5月19日

更新于

2024年7月15日

许可协议

YSOS-rust lab5 上一篇

YSOS-rust 实验报告下一篇

YSOS-rust lab4

YSOS-lab4

1. 实验要求

2. 实验过程

milestone1

milestone2

milestone3

踩坑记录

milestone4

思考题

加分项123

milestone5

加分项4

3. 关键代码

实现加载程序文件

修改boot配置

修改boot代码

修改kernel代码

实现生成用户程序

将 ELF 文件从列表中取出，并生成用户程序：

修改pkg/elf/src/lib.rs

修改GDT

实现系统调用

补全中断注册函数和dispatcher

实现中断处理函数

实现动态内存分配

实现标准输入输出

实现用户态库read_line

内核态处理输入输出

实现进程退出

实现main函数

在内核实现进程的创建和等待

实现用户态的进程创建和等待

用户态库的进程等待

系统调用支持进程创建和等待

实现main函数

内核

shell

hello

fac

实现sleep

实现UefiRuntime

实现系统调用

加分项1：记录代码段占用情况

加分项2：FrameDeallocator

加分项3：unmap_range

4. 实验结果

5. 总结

修改`pkg/elf/src/lib.rs`

补全中断注册函数和`dispatcher`

实现`main`函数