
On Linux you can bypass Mesa3D library and talk to i915, amdgpu or vc4 drivers in kernel, but then you'll need to translate your GL calls to whatever format GPU expects them to be (including shader bytecode). In conclusion - on Windows you probably cannot do anything unless you write your own kernel driver to communicate to hardware, and then you can interface with your driver. Mesa3d code to translate GL to vc4 is here: vc4 or v3d (supports newer hardware and GLES3). Raspberry Pi GPU docs are available here: VideoCore IV 3D Architecture Reference Guide. You can take a look how Linux kernel drivers communicate with actual GPU here: i915 for Intel, and amdgpu for AMD. Note that these parts of code uses bunch of other parts of mesa - like GLSL compiler. To understand what is needed to translate GL state to GPU driver format, you can use mesa3d open-source library. People have been reverse engineering older hardware, so there is some docs available, but I have no idea how complete they are: So currently nvidia driver is binary-blob only. But as far as I know, it is very incomplete.

They said they will start opening documentation here. If we talk Nvidia, then their docs are not open at all. If we go to AMD, then you can find their docs on, for example: Here' super-simple example how to do that: It does not do any 3d, but just shows what is involved in this kind of communcation. If you don't want to rewrite everything, you could reuse Intel's driver in kernel (i915) and just ignore user-space part (mesa3d) and communicate with driver directly (not GPU).

Intel has full set of documentation about how their GPU works (that's why open-source OpenGL driver in Linux is of very high quality).

But note they will be very different between vendors or even different generations of same vendor GPU's. If you want to write something directly to GPU, you could skip whole GL/D3D part and just provide commands in final GPU compatible form.

Internally it does everything to convert commands into formats GPU supports and pushes to its kernel part. The other part is user-space part that provides OpenGL, D3D and other API's for user to interact. This part usually is small, and it does not understand OpenGL, D3D or whatever other high-level API. One sits in kernel and is responsible communicating to GPU hardware. In short - there actually two parts of graphics driver in modern OS'es. There was topic here discussing same thing:
