1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
//! A library for defining CPU-local variables.
//!
//! See [`cpu_local`] for more details on how to define a CPU-local variable; the information below
//! is not required.
//!
//! # Implementation
//!
//! There are two ways in which a crate can be linked:
//! - Statically, meaning it is linked at compile-time by an external linker.
//! - Dynamically, meaning it is linked at runtime by `mod_mgmt`.
//!
//! Since we have little control over the external linker, we are forced to write CLS code
//! that works with the external linker, despite the linker having no understanding of CLS.
//! This lack of understanding results in significant complexity in calculating the offset
//! of a CLS symbol.
//!
//! Also there is no way to tell whether code was statically or dynamically linked, and so the same
//! CLS code must work with both static and dynamic linking. This results in additional complexity
//! in `mod_mgmt` because it must interface with this generic CLS code.
//!
//! ## x86-64
//!
//! ### Static linking
//!
//! On x86_64, a TLS data image looks like
//!
//! ```text
//!                        fs
//!                         V
//! +-----------------------+--------------+------------------------+
//! | statically linked TLS | TLS self ptr | dynamically linked TLS |
//! +-----------------------+--------------+------------------------+
//! ```
//! where statically linked TLS is accessed using a negative offset from the `fs`
//! register.
//!
//! Now with the way CLS works on Theseus, when we statically link CLS, the
//! linker believes we are prepending the cls section to the tls section like
//! ```
//!                        gs                            fs
//!                         V                             V
//! +-----------------------+-----+-----------------------+
//! | statically linked cls | 000 | statically linked TLS |
//! +-----------------------+-----+-----------------------+
//! ```
//! where `000` are padding bytes to align the start of the statically linked
//! TLS to a page boundary. So the linker will write negative offsets to CLS
//! relocations based on their distance from the end of the statically linked
//! TLS.
//!
//! However, in reality we have a completely separate data image for CLS, and
//! so we need to figure out the negative offset from the `gs` register based on
//! the negative offset from the `fs` register, the CLS size, the TLS size, and
//! the fact that the start of the TLS section is on the next page boundary after
//! the end of the CLS section.
//!
//! ```text
//! from_cls_start
//!    +-----+
//!    |     |
//!    |     |        -{cls}@TPOFF
//!    |     +------------------------------+
//!    |     |                              |
//!    |     |  -offset                     |
//!    |     +----------+                   |
//!    |     |          |                   |
//!    V     V          V                   V
//!    +----------------+-----+-------------+
//!    |      .cls      | 000 | .tls/.tdata |
//!    +----------------+-----+-------------+
//!    ^                ^     ^             ^
//!    |                |     |             |
//!    |                gs    |            fs
//!    |                |     |             |
//!    +----------------+     +-------------+
//!    |    cls_size          |   tls_size
//!    |                      |
//!   a*4kb                  b*4kb
//!    |                      |
//!    +----------------------+
//!     cls_start_to_tls_start
//! ```
//! where `a*4kb` means that the address is a multiple of `4kb` i.e.
//! page-aligned.
//!
//! ### Dynamic linking
//!
//! When a crate is dynamically linked on x86_64, `mod_mgmt` will set `__THESEUS_CLS_SIZE`
//! and `__THESEUS_TLS_SIZE` to `usize::MAX`. Prior to calculating the offset, the CLS
//! access code will check if the variables are set to their sentinel values and if so,
//! it will simply use the provided offset since `mod_mgmt` would have computed it
//! correctly, unlike the external linker.
//!
//! ## aarch64
//!
//! Unlike x86_64, aarch64 doesn't have different branches for statically linked, and
//! dynamically linked variables. This is because on aarch64, there are no negative
//! offset shenanigans. A TLS data image looks like
//! ```
//! fs
//! V
//! +-----------------------+------------------------+
//! | statically linked TLS | dynamically linked TLS |
//! +-----------------------+------------------------+
//! ```
//!
//! Unlike x86_64, the `.cls` section is located after `.tls` in a binary and so the
//! linker thinks the data image looks like
//! ```
//! fs
//! V
//! +-----------------------+-----+-----------------------+------------------------+
//! | statically linked TLS | 000 | statically linked CLS | dynamically linked TLS |
//! +-----------------------+-----+-----------------------+------------------------+
//! ```
//! where `000` are padding bytes to align the start of the statically linked CLS to
//! a page boundary.
//!
//! Hence, CLS symbols have an offset that is incorrect by
//! `tls_size.next_multiple_of(0x1000)`, or 0 if there is not TLS section. So, when
//! loading CLS symbols on aarch64, `mod_mgmt` simply sets `__THESEUS_TLS_SIZE` to 0.

#![no_std]

pub use cls_macros::cpu_local;

/// A trait abstracting over guards that ensure atomicity with respect to the
/// current CPU.
///
/// This trait is "sealed" and cannot be implemented by anything outside this
/// crate.
pub trait CpuAtomicGuard: sealed::Sealed {}

impl sealed::Sealed for irq_safety::HeldInterrupts {}
impl CpuAtomicGuard for irq_safety::HeldInterrupts {}

impl sealed::Sealed for preemption::PreemptionGuard {}
impl CpuAtomicGuard for preemption::PreemptionGuard {}

mod sealed {
    pub trait Sealed {}
}

#[doc(hidden)]
pub mod __private {
    pub use preemption;
    
    #[cfg(target_arch = "aarch64")]
    pub use {cortex_a, tock_registers};

    #[cfg(target_arch = "x86_64")]
    pub use x86_64;
}