The Windows NT Runtime Environment explained: Part 1

Sequence of Execution

[1] The Boot Process

There are two varieties of general-purpose computing platforms on the market: those that support the traditional BIOS interface, and those that support the more recent and featureful UEFI interface. Both are standardized firmware interfaces that provide hardware (board + peripheral) abstraction to the operating system-specific (or external third-party) bootloader during the initial stages of the bootup process. Microsoft does a good job of hiding away the nuances of the BIOS-based or UEFI-based bootup process in Windows Setup and the day-to-day Windows user experience, so the fact that there are significant differences between the two isn’t so obvious. [1]

Figure 1
Figure 1. All three stages of the Windows bootloader are firmware dependant therefore Windows installation process and boot up process will vary depending on whether the platform is BIOS-compliant or UEFI-compliant. Grey box indicates generic (non OS specific) bootloader, Blue indicates OS-specific bootloader, Orange represents the actual OS kernel.

Installation/Boot up on BIOS-compliant systems

The Windows installation process will typically format the mass storage device to one or more partitions supporting either the FAT32 or NTFS file system.* Information about the arrangement of these partitions and the file system they are associated with will be written to the very first sector of the storage medium, this is referred to as the boot sector, i.e. the fixed location where the BIOS will look to find (a) information about the medium and it’s volumes, and (b) code to execute next. In the boot sector, the BIOS will expect to find an entry in either the traditional MBR (Master Boot Record) format or of the more recent GPT (GUID Partition Table) format.** [2]

* A mass storage device may be split into many partitions. A formatted partition is referred to as a ‘physical volume’ – each physical volume will have a Volume Boot Record (VBR) written to its boot sector (the first sector) during the formatting process. An unformatted partition i.e. a partition that does not support any file system, is not considered a ‘physical volume’. See: https://www.lifewire.com/volume-vs-partition-2260237

** At the time of writing, there is only one default Windows Stage 1 bootloader for BIOS-compliant systems – and that is MBR-based; there is no support for GPT. Since a MBR is limited to a maximum volume size of 2TB, it is likely to be superseded by GPT within a few years. Meanwhile, certain hacks can be used to allow Windows to boot from a GPT medium.

Stage 1 (MBR)

After power up, when ready, the BIOS will execute the MBR’s boot code (also referred to as the ‘master boot code’) which is in essence a generic (non-operating system specific) ‘stage 1’ or ‘primary’ bootloader. Upon control being handed over to the master boot code, it functions to locate the system partition (also referred to as the active partition) so that it may load this partition’s boot code from its Volume Boot Record (VBR).

Stage 2 (VBR -> Bootmgr)

Only one volume may be flagged as the system partition in a MBR; the system partition flag indicates to the MBR’s boot code which volume’s VBR to read to find the operating-system specific ‘stage 2’ or ‘secondary’ bootloader. Note that this volume is also generally flagged as hidden (viz. System Reserved) so that it may not be seen or modified once the operating system has booted up.

In the default case of Windows Setup, a distinct 100MB NTFS partition is created and installed with Windows Boot Manager (Bootmgr) and Boot Configuration Database (BCD) files; this volume is referred to as the ‘System Volume’. The System Volume is the partition which has both system/active flag and hidden flag checked; due to the first flag, the MBR knows to hand over execution to the boot code in the System Volume’s VBR. This boot code is written during Windows Setup and its function is to (1) locate Bootmgr from the volume’s root directory, (2) load it to memory, and (3) execute/hand over control to this secondary bootloader. [3] [4, pp. 502-512] [5]

Note: Though the Volume Boot Record is written at the boot sector (viz. first sector) of all volumes during the partitioning process (to store information about the volume), not all VBRs will consist of usable boot code. Also, unlike boot code featured in the MBR, the BIOS does not execute boot code from any of the VBRs directly. [6]

For a BIOS-compliant platform, there are two executable files that constitute Bootmgr, Startup.com and Bootmgr.exe. This is due to the fact that the processor will be running in 16-bit real-mode from the time the BIOS executes the master boot code; there is no translation of virtual-to-physical memory addresses in 16-bit real-mode, and the .com executables that run in this mode rely solely upon the first 1MB of physical memory. For this reason, Startup.com is executed first and is responsible for switching the processor over to 32-bit protected-mode where full 32-bits of physical memory becomes accessible before Bootmgr.exe is executed. Bootmgr.exe has access to It is Bootmgr.exe which initializes page tables and finally enables virtual-to-physical memory address translation. [4, p. 504] [7]

Installation/Boot up on UEFI-compliant systems

The UEFI firmware is designed with the expectation that a UEFI-compliant boot manager is present in the Non-Volatile RAM (NVRAM); in the case of Windows, this is actually a distinct UEFI-compliant version of the Windows Boot Manager (Bootmgfw.efi) which is written to the NVRAM by Windows setup along with complementary BCD content. The UEFI-compliant Windows Boot Manager relies on the UEFI API and services instead of BIOS interrupts and is automatically loaded by the UEFI firmware without the need to read the boot sector of the storage medium or the relevant volume – in fact, though UEFI-MBR boot up is possible, UEFI firmware is primarily designed to support boot up from GPT mediums (UEFI-GPT boot). [4, pp. 512-513]

Note: In UEFI systems, all applications run in the native CPU mode with virtual-to-physical memory address translation/paging enabled. Images of executable UEFI API-based code (UEFI applications) generally end in the .efi file extension and can be executed whilst the UEFI “boot services” are active. UEFI boot services have to be terminated by a specific function call via the API by one of the UEFI applications – this is generally the boot manager once it has finished doing its job.

Figure 2
Figure 2. Executable code (images) involved in the Windows boot up process, and the CPU mode of execution.

References

[1] manjaro.org, “Some basics of MBR v/s GPT and BIOS v/s UEFI,” [Online].
Available: https://wiki.manjaro.org/index.php?title=Some_basics_of_MBR_v/s_GPT_and_BIOS_v/s_UEFI#MBR_vs._GPT.
[Accessed 4 March 2018].
[2] lifewire, “What is a Boot Sector?,” [Online].
Available: https://www.lifewire.com/what-is-a-boot-sector-2625815
[Accessed 4 March 2018].
[3] Microsoft, “TechNet: Master Boot Record,” [Online].
Available: https://technet.microsoft.com/en-us/library/cc976786.aspx
[Accessed 4 March 2018].
[4] M. Russinovich, D. A. Solomon and A. Ionescu, Windows Internals, Part 2 (Sixth Edition), Washington: Microsoft Press, 2012.
[5] The Starman, “The Starman’s Realm: Details of many versions of MBR and VBR,” 30 January 2018. [Online].
Available: http://thestarman.pcministry.com/asm/mbr/.
[Accessed 4 March 2018].
[6] lifewire, “What is a Volume Boot Record?,” [Online].
Available: https://www.lifewire.com/volume-boot-record-2625818.
[Accessed 4 March 2018].
[7] “Bucaro TecHelp: The Windows Bootup Process,” [Online].
Available: http://bucarotechelp.com/computers/netsafe/82001002.asp.
[Accessed 5 March 2018].

The Art of Affirmations

I was first introduced to the power and importance of affirmations in the spring of year 2015, when I would go for long walks first thing in the morning (5am!) to help my body recover from an injury.

Also during this period, I was studying immensely for my exams, and working on the AVINSoR project (A Visually Intelligent Navigation System for Application in Robotics – check out my GitHub).

Soon the long walks became 5 mile runs… and the 5 mile runs became 8 mile runs… which became the occasional half marathon.

I began these walks listening to Robin Sharma’s The Leader Who Had No Title on Audible – a book I highly recommend to anyone serious about their ambitions and becoming the best version of themselves.

The fable imparts, amongst other things, the importance of:

  • self-responsibility,
  • beliefs,
  • consideration of the use of finite time we have available on this earth,
  • routine and daily rituals.

Robin not only describes what to do, but also, how precisely to do it.

Pictures

Here are some pictures I took during my recovery.

Blean Woods, Canterbury (April 2015)
Blean Woods, Canterbury (April 2015)
Blean Woods, Canterbury (April 2015)
Blean Woods, Canterbury (April 2015)
Blean Woods, Canterbury (April 2015)

THE SECRETS

FINALLY!

Below, you will find a collection of resources that provide thorough insights into the art of affirmations.

I would suggest you view and assess all resources – to create a process that will work for you.

  • Dan Pena: You can read them out aloud to reinforce them…
    (affirmations are a positive feedback loop, you can increase their effectiveness)
  • Priest Dandapani: The 3 Ingredients of Affirmations
  • Dan Pena: consider the ‘KEYS’
    http://www.danpena.co.uk/newsletter-issue-102-15-keys-to-make-your-quantum-leap-in-2014-and-new-qla-info/
  • Jim Kwik: Important and concise information regarding identity, beliefs, and visions (with ‘motivational visuals’)

    (from Tom Bilyeu’s interview with Jim Kwik)
  • Dan Pena: Visions and Goals template to create detailed affirmations
    http://danpena.co.uk/downloads/vision_and_goals.pdf

Research: 5G C-RAN switched Ethernet fronthauls

Testbed Implementation for a 5G Cloud-Radio Access Network switched Ethernet fronthaul

As a post-graduate researcher at the University of Kent, I was responsible for leading the implementation of MAC-PHY functional split on existing open source LTE emulators, namely OpenAirInterface (June 2016 – July 2017) and srsLTE/srsENB (August – September 2017). The purpose of the implementation was to provide a configurable software foundation for a 5G Cloud-Radio Access Network (C-RAN) switched Ethernet fronthaul that was able to capture metrics. The implementation was able to produce a log file for statistical analysis, and push the captured metrics out over the network in real-time to a software-defined network (SDN) Controller for the purpose of ‘intelligent’ traffic steering.

The contributions were a part of the EU Horizon 2020 funded iCIRRUS project and the UK EPSRC (Engineering and Physical Sciences Research Council) funded NIRVANA project.

Why the shift from OAI to srsLTE/srsENB?

At the EuCNC 2017 exhibition at Oulu (Finland), I bumped into a Software Engineer from Software Radio Systems Ltd. When he told me they had recently made their software eNB implementation open source, I insisted he show me the source code… Very soon, a shift was made from OpenAirInterface to srsLTE/srsENB for two good reasons: modularity and code quality.

Publications

During my research contract, I co-authored the following research papers on the C-RAN 5G telecommunications architecture with regards to Layer 2 (MAC) Ethernet-based fronthauls:

  • An Ethernet-Based Fronthaul Implementation with MAC/PHY Split LTE Processing
    (1st author)
    Globecom 2017
  • Effects of Contention and Delay in a Switched Ethernet Evolved Fronthaul for Future Cloud-RAN Applications
    (2nd author)
    ECOC 2017
  • Ethernet-based fronthauling for cloud- and virtualized-radio access networks
    (2nd author)
    ICTON 2017

The Good Stuff

It seems like the Git hosts I used have changed things around quite a bit since I last checked, so the links below no longer work!

Bare with me as I move everything over to GitLab. Meanwhile, if interested in anything, drop me a message!

Documentation Repository
[LINK – OpenAirInterface-based splits.]

Code Repository
[LINK – OpenAirInterface/OAISIM-based MAC-PHY Split]

Faster Unsigned Integer Arithmetic

CPUs are able to perform bitwise operations on unsigned integers much faster than arithmetic operation – generally, a single bitwise operation in your source code will translate to a single machine code instruction, this instruction will be executed by a CPU core in a single clock cycle.

Here are some bitwise operations you can use in place of arithmetic:

I ÷ 2N  =  I >> N ‘right shift’ of N bits on I
(the result will always be the floor value)

I × 2N = I << N ‘left shift’ of N bits on I

I mod (N+1) = I & N bitwise ‘and’ of I and N

don’t trust anything without empirical evidence
– always test for actual performance gain in your own applications!

Microarchitecture-specific
performance optimization

It’s good practice to search the documentation of intended target architectures for SIMD (Single Instruction, Multiple Data) arithmetic blocks. These IP blocks implements logic for the sole purpose of significantly speeding up the execution of arithmetic operations on integers and floating point numbers.

These blocks will have their own CPU instruction set which you can take advantage of using the associated library provisioned by the CPU vendor.

Examples of SIMD instruction set ‘extentions’ include:

  • SSE
  • SSE2
  • SSE3
  • SSE4
  • SSE4.1
  • SSE4.2
  • SSE4a
  • POPCNT
  • LZCNT
  • AVX
  • AVX2
  • etc.

See:

  • https://www.cs.uaf.edu/2011/fall/cs301/intel_x64_megadoc_2011.pdf
  • http://www.intel.com/Assets/PDF/manual/248966.pdf
  • http://sci.tuomastonteri.fi/programming/sse
  • http://softpixel.com/~cwright/programming/simd/sse.php
  • https://felix.abecassis.me/2011/09/cpp-getting-started-with-sse/

Note: One more thing to consider is the performance advantage gained through the particular compiler being used to compile your source code for the target architecture… and the tuning and optimization parameters being passed to it!

Even without specifying tuning/optimization parameters, the compiler performance-gain may not only vary between target architectures, but also between one program and another – because of how well the compiler logic optimizes important/most-often-executed segments of the code.

See:

https://software.intel.com/en-us/forums/intel-c-compiler/topic/327585
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=13712

 

 

My thoughts on The Six Pillars of Self-Esteem – P2

Nathaniel Branden suggests that unlike other definitions of self-esteem, which are nothing more than “feel good notions” (he even lists such erroneous definitions at the end of the book with explanations on why they’re erroneous), it is the nature of our response to challenges, and the mental processes behind those responses, that affect how we see, identify, and define ourselves. This is our fundamental “sense of self”. Branden affirms that consciousness, responsibility and moral choices fundamentally affect our self-esteem, and therefore self-esteem cannot be divorced from our actions and responses.

Self-esteem is constituted of two components, which are not self-conscious thoughts that perpetuate in a high self-esteem individual, but rather, properties of the experience of self-esteem itself. The high self-esteem experience inescapably and inevitably consists:

  1. Self-efficacy:
    • “self-efficacy means confidence in the functioning of my mind, in my ability to think, understand, learn, choose, and make decisions; confidence in my ability to understand the facts of reality that fall within the sphere of my interests and needs; self-trust; self-reliance.” (Branden embodies this experience immaculately in his first-person description).
    • This experience generates the sense of control over one’s life. “The sense of being at the vital centre of one’s existence rather than a passive spectator or a victim of events”.
  2. Self-respect:
    • “self-respect means assurance of my value; an affirmative attitude toward my right to live and to be happy; comfort in appropriately asserting my thoughts, wants, and needs; the feeling that joy and fulfilment are my natural birth right.”
    • This experience generates a healthy (benevolent, non-toxic, non-neurotic) sense of community with other individuals, the sense of “fellowship of independence and mutual regard”. The polar opposite of such an experience would be either the feeling of “alienated estrangement from the human race” on one hand, or “mindless submergence into a tribe” on the other.

Nathaniel Branden continually stresses the importance of working on self-esteem with clarity and focus. A precise and distinguishing definition, understanding and comprehension of what is implied by ‘self-esteem’ is required – so that it is not confused with anything that it is not.

It’s at this point that I’m reminded of another great book I’m listening to, Genghis Khan: And the Making of the Modern World, which gives a remarkable historic narrative of the early childhood of Genghis Khan, whose mother Hoelun is abandoned by the tribe after the death of her husband (Genghis Khan’s father) Yesügei Baghatur.  Yesügei is treacherously poisoned by the Tartars whilst sharing a meal with them, so with this fate, Genghis Khan and his six siblings are raised along the cold and arid banks of the Onon river. Being one of the eldest, Genghis would help his mother forage for roots, berries and millets, and would later learning to hunt rats and other small rodents when they would become available. Should his circumstances be considered unfortunate? It is this very experience that forged his sense of self-efficacy, and his trust in his own ability to think and come up with the ingenious solutions to practical problems of raid and warfare that were vital to the sustenance of the nomadic tribes of arid lands. Author Professor Jack Weatherford later gives a description of the remarkable way in which Genghis Khan, with his nomadic tribesman (all on horseback), lay siege to Bukhara and Samarkand. Genghis Khan seeks to put the arrogant sultan Shah Muhammad and his vicious governors to justice – to avenge the disfigurement of his envoys delivering nothing more than a sincere letter initiating a friendship and a trade deal. Throughout the book it also becomes clear how Genghis Khan, through the experience that circumstances afford him, copes with challenges – only to become rooted in and affirmative of his own individual values, well before he is even titled “The Great Khan”.

My thoughts on The Six Pillars of Self-Esteem – P1

I recently started reading ‘The Six Pillars of Self-Esteem‘ by psychotherapist Nathaniel Branden (after reading so many reviews and recommendations in so many different places). Here’s what I’ve discerned from this great book so far…

The cover just wouldn’t stay down without my pen on it. No pun intended.

 

Continue reading My thoughts on The Six Pillars of Self-Esteem – P1