diff --git a/Practicals/RUNBOOK.md b/Practicals/RUNBOOK.md new file mode 100644 index 0000000..bd73e63 --- /dev/null +++ b/Practicals/RUNBOOK.md @@ -0,0 +1,553 @@ +# EMBS runbook - Vivado -> Vitis HLS -> Vitis -> board + +Personal quick-reference for the assessment. Compiled from the EMBS docs at +. Running example: the **Game of Life** +(Practical 3b), because that is the one you already have working. + +--- + +## Read this first: three corrections to the step order + +You listed eleven steps. Eight are right. Two are in the wrong place, and one +was ambiguous. Everything below follows the corrected order. + +**1. "Application which uses the HLS component" cannot come before "wiring the +HLS IP into the block diagram."** The driver (`xtoplevel.h`), the base addresses +(`xparameters.h`) and the `XToplevel_*` functions are all *generated from the +exported XSA*. Until the IP is in the block design, a bitstream is built and the +hardware is re-exported, those files do not exist. Writing the app first means +writing against nothing. + +**2. "Simulating this" is not a separate step, and the docs only cover one kind.** +See the note below. + +**3. You build the bitstream twice, and they are different things.** The first +(step 2) validates the base system and is what creates the platform. The second +(step 7) is the real one, containing your IP. Expect to throw the first away. + +### The corrected order + +| # | Step | Where | +| --- | --- | --- | +| 1 | Create the block design | P1a | +| 2 | Build the **base** bitstream | P1a | +| 3 | Export hardware, create the **Platform**, first app | P1a | +| 4 | Write the **HLS component** | P3a | +| 5 | Write the **testbench**, run **C Simulation** | P3a | +| 6 | **Wire the IP into the block design** | P4 | +| 7 | Build the bitstream **again**, re-export the XSA | P4 | +| 8 | **Re-read the XSA**, then write the app that drives the IP | P4 | +| 9 | **Program the board** and run | P1a / P4 | +| 10 | **Test it out** | P4 / Software API | + +### About "simulating this" + +Three different things get called simulation, and the docs only document the +first: + +- **HLS C Simulation** - runs your testbench as ordinary C++ in software. This + is step 5, and it is the only simulation the practicals use. It checks logic, + **not** speed or hardware size. +- **C/RTL co-simulation** - re-runs the testbench against the *generated RTL*. + More thorough, much slower. Not covered anywhere in the EMBS docs. +- **Vivado behavioural simulation** - simulates the whole block design. Not + covered anywhere in the EMBS docs. + +So: do step 5 properly, and treat the FPGA as the only other test rig you have. + +--- + +## Part 0 - Once per session + +On the CS servers (or your own machine) Vivado and HLS need the licence server +and the tools on the PATH: + +```bash +echo 'export LM_LICENSE_FILE=2100@cslm0.its.york.ac.uk' >> ~/.bashrc +echo 'export PATH=/opt/york/cs/net/xilinx_vivado-2025.1-x86_64-1/2025.1/Vivado/bin:$PATH' >> ~/.bashrc +``` + +Reopen the terminal. Check `vivado -version` says **2025.1**, not 2020.x. + +Verify the licence is visible: **Help -> Manage License -> Manage License Search +Paths** should show `LM_LICENSE_FILE = 2100@cslm0.its.york.ac.uk`. + +The three tools, in one line each: + +- **Vivado** - builds the hardware design (the bitfile) and programs the FPGA. +- **Vitis** - builds software for the ARM cores, and loads both. +- **Vitis HLS** - turns C/C++ into a custom IP core you can drop into Vivado. + +> **Never put spaces** in a Vivado project path, a Vitis workspace, or a project +> name. It genuinely breaks the tools later. + +--- + +## Part 1 - Base hardware and platform (Practical 1a) + +### 1.1 New project + +1. Vivado -> **Create Project**. +2. Name and location (**no spaces**). +3. **RTL Project** + "Do not specify sources at this time". +4. Next past Constraints. +5. Default Part page -> **Boards** -> Vendor `digilentinc.com`, Name + `Zybo Z7-10`. First time only: **Refresh** to pull the board catalogue, then + click the download arrow in the Status column to install the board files. +6. Select **Zybo Z7-10** -> Next -> Finish. + +### 1.2 Block design + +1. Flow Navigator -> **Create Block Design** (default name `design_1` is fine). +2. **Add IP** -> `ZYNQ7 Processing System`. +3. Switch the centre panel to **Board** -> double-click **4 LEDs** -> OK. + Then **4 Buttons** -> OK. (This drops in an `axi_gpio` core.) +4. Green banner -> **Run Connection Automation** -> OK. + This adds a *Processor System Reset*, an *AXI Interconnect*, and wires the PS + master `M_AXI_GP0` through to the GPIO slave `S_AXI`. +5. Green banner -> **Run Block Automation** -> OK. This attaches DDR and the + fixed I/O, and sets clock speeds. +6. Toolbar -> **Regenerate Layout** to tidy the diagram. + +### 1.3 Wrapper and bitstream + +1. **Sources** panel -> right-click `design_1.bd` -> **Create HDL Wrapper** -> + "Let Vivado manage wrapper and auto-update" -> OK. +2. A critical warning about `PCW_UIPARAM_DDR_DQS_TO_CLK_DELAY_*` is **expected and + ignorable**. Any *other* critical warning is not. +3. Flow Navigator -> **Generate Bitstream** -> Yes to run synthesis and + implementation. Minutes for this design; much longer later. +4. Progress: **Design Runs** tab at the bottom. + +Worth a look once, for the assessment: **Open Implemented Design** -> Device view, +**Report Utilization**, **Open Synthesized Design** -> **Schematic**. + +### 1.4 Program the FPGA (base design) + +1. Flow Navigator -> **Open Hardware Manager** -> **Open Target** -> **Open New + Target**. +2. Right-click `xc7z010` -> **Program Device**. +3. Bitstream should already be `design_1_wrapper.bit` -> **Program**. + +The green DONE LED lights, **and nothing else happens** - the ARM cores have no +software yet. + +### 1.5 Platform, then first application + +1. Vivado -> **File -> Export -> Export Hardware** -> Next -> **Include + bitstream** -> Next -> Finish. This produces `design_1_wrapper.xsa`. +2. **Tools -> Launch Vitis IDE** -> OK. +3. Vitis -> **File -> Set Workspace** (e.g. `~/vitis_workspace`, no spaces). +4. **File -> New Component -> Platform**. Name it e.g. `basicIO_platform`. +5. Browse to the exported **`design_1_wrapper.xsa`**. +6. Defaults are correct: `standalone` and `ps7_cortexa9_0` -> Next -> Finish. +7. **View -> Examples -> Embedded Software Examples -> Hello World -> Create + Application Component From Template**. Pick your platform, then the + `standalone_ps7_cortexa9_0` domain. + +Vitis project types, because they get confused: + +- **Platform** - the exported hardware plus drivers. Comes from Vivado. +- **System** - generated; drivers/frameworks your apps sit on ("the OS"). +- **Application** - your code. + +**You must make a new System whenever the hardware interface changes.** + +--- + +## Part 2 - The HLS component and its testbench (Practical 3a/3b) + +### 2.1 New HLS component + +1. Vitis -> **File -> New Component -> HLS**. +2. Name it descriptively (not `hls_component`). +3. Configuration File -> Next. Add Source Files -> Next. +4. **Part = `xc7z010clg400-1`**. Next -> Next -> Finish. +5. Add your sources (`gameoflife.cpp`, `gameoflife.h`). +6. **Settings -> hls_config.cfg** -> under *C Synthesis sources* set + **top = `game_of_life_step`**. + +Your existing `hls_config.cfg` already has the right shape: + +``` +part=xc7z010clg400-1 +[hls] +flow_target=vivado +package.output.format=ip_catalog +package.output.syn=false +tb.file=testbench.cpp +syn.file=gameoflife.cpp +syn.file=gameoflife.h +syn.top=game_of_life_step +``` + +### 2.2 The interface template + +Two interfaces, and they do different jobs: + +- **AXI Slave (`s_axilite`)** - the ARM uses it to start/stop the core and to + pass a few scalars. +- **AXI Master (`m_axi`)** - the core uses it to read/write **main memory** + directly. Only needed if you are moving a lot of data. + +The Game of Life packs its whole 10x10 grid into 4 x 32-bit words, so it needs +**only the slave interface**: + +```c +#pragma HLS INTERFACE s_axilite port = in_words bundle = AXILiteS +#pragma HLS INTERFACE s_axilite port = out_words bundle = AXILiteS +#pragma HLS INTERFACE s_axilite port = return bundle = AXILiteS +``` + +If you *do* use an AXI Master, the template is: + +```c +#pragma HLS INTERFACE m_axi port=ram offset=slave bundle=MAXI +#pragma HLS INTERFACE s_axilite port=ram bundle=AXILiteS +#pragma HLS INTERFACE s_axilite port=return bundle=AXILiteS register +``` + +> **Trap.** The knowledge base uses `register` on the slave ports; Practical 3a's +> template omits it. The docs are explicit that the `port=return` pragma is what +> bundles **start / stop / done / reset** into a control register: *"If you do not +> include this pragma, then HLS will generate simple wires for these signals +> instead and your IP core will not be controllable from the ARM cores directly."* +> Include it, or you get an IP you cannot start. + +> **Trap.** For an `m_axi` port the datatype must be **32-bit** (`uint32`), because +> RAM is 32 bits wide. Returning `uint100 toplevel(uint100 *ram)` makes HLS +> generate 100 wires that you cannot wire up in Vivado. + +### 2.3 Testbench and C Simulation + +1. Right-click **Test Bench** -> **New Test Bench File** -> `testbench.cpp`. +2. **Settings -> hls_config.cfg -> Testbench Sources -> Add File** and add it. + (Creating the file is *not* enough - this is the step people miss.) +3. Flow panel -> **C Simulation -> Run**. + +Good testbench habits, from the docs: return **0 on success, 1 on failure**, +check against pre-calculated values, and for a design like the Game of Life +**run several generations in sequence** so you know state carries over correctly. +Edge behaviour (wrap-around or not) is your choice - but decide, and test it. + +### 2.4 Synthesis and tuning + +Flow panel -> **C Synthesis -> Run**. Then read the report: + +- **Timing Estimate** - at 100 MHz the target period is 10 ns. The 3a design + estimates ~7.3 ns, so it meets timing. +- **Performance and Resource Estimates** - overall latency and interval, then + per-loop. This is why loops are **labelled** in EMBS code. +- **Utilisation** - LUTs, FFs, DSPs, BRAMs. Click the % toggle to see it as a + fraction of the device. The Z-7010 has **60 BRAMs**, 36 kbit each. + +The 3a worked example is the clearest demonstration of the trade-off: + +| Change | Latency | Notes | +| --- | --- | --- | +| Baseline (with `PIPELINE off` deliberately) | 910 | | +| `PIPELINE` on both loops | 521 | Interval 1; trip counts 100 and 99 | +| `UNROLL` both loops | ~420 | ~7300 LUTs - **3x the size**; HLS used 4 BRAMs instead of 1 | +| + `INLINE` on both functions | 366 | Saved hardware by scheduling across functions | +| + `ARRAY_PARTITION type=complete` on the array, `UNROLL` the read loop | **115** | BRAMs down, LUTs even slightly down | + +Inspect with **C Synthesis -> Reports -> Schedule Viewer** and the **Analysis** +perspective. Columns are clock states; things in the same column happen in +parallel. A successfully unrolled design looks *vertical*; a stuck one looks +*horizontal*. + +**BRAMs only have two access ports**, so an array in BRAM caps your parallelism +at 2. `ARRAY_PARTITION` with `type=complete` moves it into registers. For a +multidimensional array, set **`dim=0`** so all dimensions are partitioned. + +Game of Life targets (P3b): **latency under 300 cycles**; ~200-250 is reachable; +the author has one at **43 cycles**. + +Useful directives, from the knowledge base: `PIPELINE`, `UNROLL`, `INLINE`, +`ARRAY_PARTITION`, `ARRAY_MAP`, `ALLOCATION`, `LATENCY`, `LOOP_FLATTEN`, +`LOOP_TRIPCOUNT`, `DATAFLOW`, `RESOURCE`, `BIND_STORAGE`. + +--- + +## Part 3 - Put the IP into the hardware (Practical 4) + +### 3.1 Export the IP + +In Vitis HLS -> Flow Navigator -> **Package**: + +- **output.file** = something sensible (e.g. `practical4`) +- **output.format** = `ip_catalog` +- **Run** + +### 3.2 Add it to Vivado + +1. Vivado -> open your Block Design. +2. **Window -> IP Catalog**. +3. Flow Navigator -> **Settings** -> **IP** -> **Repository** -> **+**. +4. Select your **HLS project directory** -> Select. Vivado scans and reports an + IP was added -> OK -> OK. +5. In the Block Design -> **Add IP** -> your core is listed under the Display + Name you set (or "Toplevel" if you forgot). + +### 3.3 Wire it up + +**Only if your IP has an `m_axi` port** (it needs main memory): + +1. Double-click the **Zynq** block -> **PS-PL Configuration** -> expand + **HP Slave AXI Interface** -> tick **S AXI HP0** -> OK. A new port appears. +2. **Run Connection Automation** -> tick **S_AXI_HP0** -> it offers to connect it + to your IP's `m_axi`. + +**Always:** + +3. In the same Connection Automation dialog, tick **`s_axi_AXILiteS`** -> it + offers to connect to **`M_AXI_GP0`** on the processing system -> OK. + +Result: your slave port goes through `processing_system7_0_axi_periph` to the PS, +and (if you have one) your master port goes through `axi_mem_intercon` into the +Zynq. + +> The Game of Life as written has **no `m_axi` port**, so it needs **no S AXI +> HP0** - just the GP0 path. Don't tick HP0 unless you actually have a master +> interface. + +**Tracing rule to remember:** the Zynq's `M_AXI` must be traceable to every +`S_AXI` in your design, and any IP's `M_AXI` must be traceable to `S_AXI_HP0` on +the Zynq. Vivado may use an **AXI SmartConnect** instead of an AXI Interconnect - +that is fine. + +If the AXI structure comes out wrong: delete all the AXI Interconnect blocks and +re-run Connection Automation. + +### 3.4 Rebuild and re-export + +1. Save the block design. +2. **Generate Bitstream**. +3. **File -> Export -> Export Hardware** -> **Include bitstream** -> + **overwrite** the existing XSA. + +--- + +## Part 4 - Application, programming, testing + +### 4.1 Re-read the XSA, then write the app + +**Order matters here - this is the step that was out of place in your list.** + +1. In Vitis, open your **platform**'s `vitis-comp.json` -> + **Switch / re-read XSA** -> point at the new XSA. +2. Under **drivers**, your IP now appears (e.g. `toplevel`). +3. **File -> New Component -> Application**, on the `design_1_wrapper` + platform, **letting it create a new System**. Required whenever the hardware + interface changes. + +### 4.2 Driving the IP from the ARM + +The core principle, from the Software API page: **you never "pass data" to HLS.** +The data is in shared main memory; the ARM just tells the core where to look. + +```c +#include +#include "xil_printf.h" +#include "xparameters.h" +#include "xtoplevel.h" +#include "xil_cache.h" + +u32 shared[1000]; + +int main() { + XToplevel hls; + Xil_DCacheDisable(); + + for (int i = 0; i < 100; i++) shared[i] = i; + shared[0] = 8000; + + XToplevel_Initialize(&hls, XPAR_TOPLEVEL_0_BASEADDR); + XToplevel_Set_ram(&hls, (u32) shared); // where the core should look + XToplevel_Start(&hls); + while (!XToplevel_IsDone(&hls)); + + printf("arg2 = %lu\narg3 = %lu\n", + XToplevel_Get_arg2(&hls), XToplevel_Get_arg3(&hls)); + return 0; +} +``` + +**Always set the RAM offset.** `XToplevel_Set_ram` is what makes `ram[0]` inside +HLS mean "start of my shared array" rather than "address 0 of DDR". + +Getter/setter names change with direction. If the core only *reads* an argument +you get `XToplevel_Set_arg1` / `XToplevel_Get_arg1`; if it reads **and writes** +you get `XToplevel_Set_arg1_V_i` / `XToplevel_Get_arg1_V_o`. Check the generated +`xtoplevel.h` rather than guessing. + +### 4.3 Caches - the silent wrong-answer bug + +The ARM's caches are invisible to your IP, and HLS's writes do not invalidate +them. You must handle it explicitly: + +```c +#include + +// after writing shared data, before starting the core +Xil_DCacheFlush(); +// or: Xil_DCacheFlushRange((INTPTR) shared, sizeof(shared)); + +XToplevel_Start(&hls); +while (!XToplevel_IsDone(&hls)); + +// before reading results back +Xil_DCacheInvalidate(); +// or: Xil_DCacheInvalidateRange((INTPTR) shared, sizeof(shared)); +``` + +`Xil_DCacheDisable()` also works and is simpler - just slower. + +### 4.4 Fix the bitstream path before you iterate + +Do this **once**, before you start changing HLS code. Vitis does not reliably +refresh its own copy of the bitfile, so it can silently program **old hardware**. + +1. Application -> **Settings -> launch.json**. +2. **Bitstream File** is probably + `${workspaceFolder}//_ide/bitstream/design_1_wrapper.bit`. +3. Change it to Vivado's real output: + `/.runs/impl_1/design_1_wrapper.bit`. + +### 4.5 Program and run + +From Vitis, hitting **Run** does both halves: programs the **PL** with the +bitstream, and loads the **ARM executable** into DDR. That is why "upload to +hardware" is really two uploads. + +### 4.6 Test it out + +- **Serial output** on a lab PC: `screen /dev/ttyUSB1 115200` (try ttyUSB0/2 if + that fails). Exit with **Ctrl-A**, release, then **k**, then **y**. + Full settings: 115200 8N1, no flow control. +- **Do not use the SDK/Vitis built-in console** - it is line-buffered and gets + slow with lots of data. +- For **single keypresses** use `XUartPs_IsReceiveData` / + `XUartPs_RecvByte` / `XUartPs_SendByte` from `xuartps_hw.h`, because `scanf` + blocks until a newline. +- **Timing** measurements: `XTime_GetTime`, with `COUNTS_PER_SECOND`, from + `xiltimer.h`. Put a `usleep(1000)` first or the timer reads 0. +- **Debug your own IP**: add a `version` slave argument set to a constant (e.g. + 42). If `XToplevel_Get_version()` returns 42, the slave interface and the whole + build/export chain are good. Then add a `mode` argument and switch between + normal operation and small AXI read/write test cases - that isolates a broken + master interface from a broken algorithm. + +--- + +## The loop: when you change your HLS + +This is the sequence you will actually live in. The docs list it in two places +(Practical 4 and the knowledge base) and it is the same list: + +1. Re-run **synthesis** in HLS. +2. **Re-export** the IP core (Package -> Run). +3. In Vivado, it should say **"IP Catalog is out-of-date"**. If not: **IP Status** + -> **Re-Run Report**, then **Refresh IP Catalog**. +4. In the **IP Status** panel, select your IP -> **Upgrade Selected**. +5. **Generate Output Products** -> **Generate**. +6. **Generate Bitstream**. +7. **Export Hardware** (include bitstream). +8. In Vitis, platform -> **Settings -> vitis-comp.json -> Switch / re-read XSA**. +9. Reprogram and run. +10. If you changed the **interface**, regenerate the System and move the + application into it. + +> Step 8 is not optional even with the launch.json fix. Re-reading the XSA is what +> updates the hardware description, the **drivers** and **xparameters.h**. The +> launch.json change only fixes *which bitfile gets programmed*. + +**Symptom of getting this wrong:** software hangs talking to your core, or the +core returns stale/nonsense results, straight after a change that "should" have +worked. Compare modification times: + +``` +//_ide/bitstream/design_1_wrapper.bit <- Vitis' copy +/.runs/impl_1/design_1_wrapper.bit <- what Vivado built +``` + +If the first is older, Vitis is running old hardware. + +--- + +## Quick reference + +| I want to... | Where | +| --- | --- | +| Set the HLS top function | HLS: Settings -> `hls_config.cfg` -> C Synthesis sources -> top | +| Register a testbench | HLS: Settings -> `hls_config.cfg` -> Testbench Sources -> Add File | +| Run the testbench | HLS: Flow -> C Simulation -> Run | +| See latency / II | HLS: C Synthesis -> Reports -> Synthesis | +| See what runs in parallel | HLS: C Synthesis -> Reports -> Schedule Viewer, and the Analysis perspective | +| Add a directive | HLS: View -> HLS Directives (or the two-star button) | +| Package the IP | HLS: Flow Navigator -> Package -> `ip_catalog` -> Run | +| Register the IP repo | Vivado: Settings -> IP -> Repository -> + | +| Enable DDR access | Vivado: Zynq block -> PS-PL Configuration -> HP Slave AXI -> S AXI HP0 | +| Auto-wire AXI | Vivado: green banner -> Run Connection Automation | +| Export hardware | Vivado: File -> Export -> Export Hardware -> Include bitstream | +| Re-read hardware in Vitis | Platform -> Settings -> `vitis-comp.json` -> Switch / re-read XSA | +| Fix which bitfile is programmed | Application -> Settings -> `launch.json` -> Bitstream File | +| Import a driver example | Platform -> `vitis-comp.json` -> ... -> Board Support Package -> drivers -> Import Examples | +| Measure code time | `XTime_GetTime` from `xiltimer.h` | + +--- + +## Traps that cost hours + +1. **The `port=return` pragma with `register`.** Without it your IP has no + control register and cannot be started from the ARM. +2. **Forgetting to add the testbench to `hls_config.cfg`.** Creating the file is + not enough. +3. **Forgetting `XToplevel_Set_ram`.** Your core writes over arbitrary memory. +4. **Caches.** Flush before the core starts, invalidate before reading results. +5. **Not re-reading the XSA.** Stale drivers and stale `xparameters.h`. +6. **Vitis programming an old bitfile.** Fix the launch.json path once. +7. **Spaces in paths or project names.** +8. **A `uint32`-vs-wide datatype on an `m_axi` port.** 100 wires you cannot route. +9. **Running out of disk.** Check `quota -s`; aim for ~500 MB free. `~/.cache` is + the usual culprit. If the tools misbehave after filling the disk, delete + `.Xilinx` and `.Xil` from your home directory. +10. **Not making a new System after changing the interface.** The application + will not see the new driver. + +--- + +## VLAB notes (you are off campus) + +Everything above works unchanged, with three differences: + +- **Program the FPGA** via **Remote Server** rather than a local target: + Hostname `localhost`, Port `12345` (whatever you passed to `vlab.py -l`). + In Vivado: Open Hardware Manager -> Open Target -> Open New Target -> Remote + Server. In Vitis: Settings -> `launch.json` -> Target Connection -> new. +- **Serial** - you do not need `screen /dev/ttyUSB*`. The VLAB client *is* the + serial terminal. +- **No buttons, no LEDs, no live HDMI.** You cannot observe anything except over + the UART. Design for that: this is exactly why the Game of Life testbench and + the `version`/`mode` debug arguments matter so much more on the VLAB. + +Remember `vlab.py` must run **on the machine running the Xilinx tools**, and on +a shared server you need your own port (`-l 20041`) because 12345 collides. + +--- + +## Where each part came from + +| Topic | Page | +| --- | --- | +| Project, block design, bitstream, platform, first app | `docs/practicals/practical-1a` | +| GPIO, UART calculator, extending the hardware | `docs/practicals/practical-1b` | +| HLS project, testbench, synthesis, directives | `docs/practicals/practical-3a` | +| Game of Life spec and targets | `docs/practicals/practical-3b` | +| Exporting IP, wiring AXI, iteration loop, Collatz | `docs/practicals/practical-4` | +| Sharing memory, caches, memcpy, timers, UART, Ethernet | `docs/resourcestips/software-api` | +| Terms, directives, AXI interfaces, loop optimisations | `docs/resourcestips/hls-knowledge-base` | +| Debug interfaces and debug modes for your own IP | `docs/resourcestips/debugging` | +| Serial terminal settings | `docs/resourcestips/uart` | +| Disk quota and cleaning projects | `docs/resourcestips/disk-usage` | +| VLAB remote target and serial | `docs/setup/vlab`, `docs/setup/vlab-quickstart` |