Hi, Habr. Here I have prepared for you a small guide about NTFS Reparse points (hereinafter RP). This article is for those who are just starting to dive into the Windows kernel drivers development. In the beginning, I will explain the theory with examples, then I will give an interesting task to solve.
RP is one of the key features of the NTFS file system, which can be useful in solving backup and recovery tasks. As a result, Acronis is very interested in this technology.
If you want to figure out the topic yourself, check out these resources. The theoretical part that you will see next is a summary of materials from this list.
A Reparse Point (RP) is an object of a given size with programmer-defined data and a unique tag. The custom object represented by the structure REPARSE_GUID_DATA_BUFFER.
The RP data block size is up to 16 kilobytes.
ReparseTag — 32-bit tag
ReparseDataLength — data size
DataBuffer — pointer to user data
RPs provided by Microsoft is represented by the structure REPARSE_DATA_BUFFER. It should not be used for custom RPs.
The format of the tag:
M — Reserved bit by Microsoft; If this bit is set, then the tag was developed by Microsoft.
L — Delay bit; If this bit is set, then the data referenced by the RP is located on the medium with a slow response speed and a long data output delay.
R — Reserved bit;
N — Name change bit; If this bit is set, then the file or directory represents another named entity in the file system.
Tag value — Must be requested from Microsoft;
Each time the application creates or deletes an RP, NTFS updates the \\$Extend\\$Reparse metadata file where RP records are stored. This centralized storage allows any application to sort and efficiently search for the desired object.
Using RP the Windows provides support for symbolic links, remote storage systems, and mount points for volumes and directories.
By the way, hard links in Windows are not an actual object, but simply a synonym to the same file on disk. These are not separate filesystem objects, but simply another file name in the file location table. This is how hard links differ from symbolic links.
To use RP, we need to write:
1. Acquiring the necessary privileges
2. Preparing the structure REPARSE_GUID_DATA_BUFFER. In our example, we will write a simple string “My reparse data” to the RP data, but it can be more valuable data.
3. Creating the file.
4. Filling our file with our structure, using the function DeviceIoControl with parameter FSCTL_SET_REPARSE_POINT.
Full source code of this app could be found here.
After build and run we get the file. Utility fsutil could help to look into the file we created and make sure that our data is in place.
It's time to look at this file from the kernel side. I will not go into the details of the mini-filter driver development. There is a good explanation in the official documentation from Microsoft with code examples. Instead we'll take a look at the post callback method.
We need to re-request IRP with the FILE_OPEN_REPARSE_POINT parameter. To do this, we'll call FltReissueSynchronousIo. This function will repeat the request, but with updated Create.Options field.
Inside the structure PFLT_CALLBACK_DATA there is a TagData. If we call FltFsControlFile with FSCTL_GET_REPARSE_POINT parameter, we will get our buffer with data.
Then we can use this data depending on the task. We can re-request the IRP. Or initiate a completely new request. For example, the LazyCopy project stores the path to the original file in the RP data. The author does not start copying when the file is opened, but only re-saves the data from RP into the stream context of this file. Data starts to be copied the moment the file is read or written. Here are the highlights of his project:
RPs have a wide range of uses and open up many possibilities for solving various problems. We will analyze one of them by solving the following task.
The game Half-life can run in two modes: software mode and hardware mode, which differ in the way graphics are rendered in the game. After digging in a little with IDA Pro, we can see that the modes differ by the loaded library sw.dll or hw.dll using method LoadLibrary.
Depending on the input arguments (for example, “-soft”), this or that string is selected and passed to the function call LoadLibrary.
The goal of the task is to load the game only in hardware mode without the user noticing it. Ideally, so that the user does not even realize that regardless of his choice, that the game is loaded in hardware mode.
Of course, it would be possible to patch the executable file or replace the dll file, or even just copy hw.dll and rename the copy to sw.dll, but we are not looking for easy ways. Plus, if the game is updated or reinstalled, the effect will disappear.
I suggest the following solution: write a small mini-filter driver. It will run continuously and will not be affected by reinstalling and updating the game. Let's register the driver for the IRP_MJ_CREATE operation, because every time the executable calls LoadLibrary, it essentially opens the library file. As soon as we notice that the game process is trying to open the sw.dll library, we will return the STATUS_REPARSE status and ask to repeat the request, but this time to open hw.dll. Result: the library we need was opened, even though the user-space asked for another one.
First of all, we need to understand what process is trying to open the library, because we need to turn our trick only for the game process. To do this, right in DriverEntry we will need to call PsSetCreateProcessNotifyRoutine and register a method that will be called every time a new process appears in the system.
In this method, we must get the name of the executable file. We could use the function ZwQueryInformationProcess.
If the name matches the target name, in our case it is hl.exe, we have to save its PID.
So, now we have the PID of the process of our game saved in the global object. We can move to pre create callback. There we should get the name of the file that is trying to be opened. FltGetFileNameInformation will help us with this. This function cannot be called at the DPC interrupt level (read more in about IRQL), however, we are going to make a call at pre create, which guarantees us a level not higher than APC.
Then, if our name is sw.dll, then we need to replace it in FileObject with hw.dll. And return the status STATUS_REPARSE.
Of course, the implementation of the project as a whole is a bit more complex, but I tried to reveal the main points. The whole project with details is here.
To simplify our test runs, instead of a game, we will run a small application and libraries with the following content:
Let's build testapp.exe and link with testdll0.dll, copy them to the virtual machine, and also prepare testdll1.dll. The task of our driver will be to replace testdll0 with testdll1. We will understand that we have succeeded if we see the message “hello from test dll 1” in the console instead of “hello from test dll 0”. Let's run it without a driver to make sure our test application works correctly:
Now let's install and run the driver:
Running the same application, we will get a completely different output to the console since another library was loaded:
Plus, in the application written for the driver, we see logs that say that we really caught our open request and replaced one file with another. The test was successful, it's time to test the solution on the game itself.
I hope it was helpful and interesting.
RP is one of the key features of the NTFS file system, which can be useful in solving backup and recovery tasks. As a result, Acronis is very interested in this technology.
Useful links
If you want to figure out the topic yourself, check out these resources. The theoretical part that you will see next is a summary of materials from this list.
- Official documentation from MSDN
- P. Yosifovich, A. Ionescu, M. Russinovich and D. Solomon, Windows Internals. Part 2, 6th Edition.
- A short and convenient guide (sorry only in russian)
- Very cool project that uses RP
- The solution to the problem that we will analyze below
A bit of theory
A Reparse Point (RP) is an object of a given size with programmer-defined data and a unique tag. The custom object represented by the structure REPARSE_GUID_DATA_BUFFER.
typedef struct _REPARSE_GUID_DATA_BUFFER {
ULONG ReparseTag;
USHORT ReparseDataLength;
USHORT Reserved;
GUID ReparseGuid;
struct {
UCHAR DataBuffer[1];
} GenericReparseBuffer;
} REPARSE_GUID_DATA_BUFFER, *PREPARSE_GUID_DATA_BUFFER;
The RP data block size is up to 16 kilobytes.
ReparseTag — 32-bit tag
ReparseDataLength — data size
DataBuffer — pointer to user data
RPs provided by Microsoft is represented by the structure REPARSE_DATA_BUFFER. It should not be used for custom RPs.
The format of the tag:
M — Reserved bit by Microsoft; If this bit is set, then the tag was developed by Microsoft.
L — Delay bit; If this bit is set, then the data referenced by the RP is located on the medium with a slow response speed and a long data output delay.
R — Reserved bit;
N — Name change bit; If this bit is set, then the file or directory represents another named entity in the file system.
Tag value — Must be requested from Microsoft;
Each time the application creates or deletes an RP, NTFS updates the \\$Extend\\$Reparse metadata file where RP records are stored. This centralized storage allows any application to sort and efficiently search for the desired object.
Using RP the Windows provides support for symbolic links, remote storage systems, and mount points for volumes and directories.
By the way, hard links in Windows are not an actual object, but simply a synonym to the same file on disk. These are not separate filesystem objects, but simply another file name in the file location table. This is how hard links differ from symbolic links.
To use RP, we need to write:
- A small application with the privileges SE_BACKUP_NAME or SE_RESTORE_NAME, which will create a file containing the RP structure, set the required ReparseTag field and fill the DataBuffer
- A kernel-mode driver that will read buffer data and handle calls to this file.
Creating our own file with RP
1. Acquiring the necessary privileges
void GetPrivilege(LPCTSTR priv)
{
HANDLE hToken;
TOKEN_PRIVILEGES tp;
OpenProcessToken(GetCurrentProcess(),
TOKEN_ADJUST_PRIVILEGES, &hToken);
LookupPrivilegeValue(NULL, priv, &tp.Privileges[0].Luid);
tp.PrivilegeCount = 1;
tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED;
AdjustTokenPrivileges(hToken, FALSE, &tp,
sizeof(TOKEN_PRIVILEGES), NULL, NULL);
CloseHandle(hToken);
}
GetPrivilege(SE_BACKUP_NAME);
GetPrivilege(SE_RESTORE_NAME);
GetPrivilege(SE_CREATE_SYMBOLIC_LINK_NAME);
2. Preparing the structure REPARSE_GUID_DATA_BUFFER. In our example, we will write a simple string “My reparse data” to the RP data, but it can be more valuable data.
TCHAR data[] = _T("My reparse data");
BYTE reparseBuffer[sizeof(REPARSE_GUID_DATA_BUFFER) + sizeof(data)];
PREPARSE_GUID_DATA_BUFFER rd = (PREPARSE_GUID_DATA_BUFFER) reparseBuffer;
ZeroMemory(reparseBuffer, sizeof(REPARSE_GUID_DATA_BUFFER) + sizeof(data));
// {07A869CB-F647-451F-840D-964A3AF8C0B6}
static const GUID my_guid = { 0x7a869cb, 0xf647, 0x451f, { 0x84, 0xd, 0x96, 0x4a, 0x3a, 0xf8, 0xc0, 0xb6 }};
rd->ReparseTag = 0xFF00;
rd->ReparseDataLength = sizeof(data);
rd->Reserved = 0;
rd->ReparseGuid = my_guid;
memcpy(rd->GenericReparseBuffer.DataBuffer, &data, sizeof(data));
3. Creating the file.
LPCTSTR name = _T("TestReparseFile");
_tprintf(_T("Creating empty file\n"));
HANDLE hFile = CreateFile(name,
GENERIC_READ | GENERIC_WRITE,
0, NULL,
CREATE_NEW,
FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS,
NULL);
if (INVALID_HANDLE_VALUE == hFile)
{
_tprintf(_T("Failed to create file\n"));
return -1;
}
4. Filling our file with our structure, using the function DeviceIoControl with parameter FSCTL_SET_REPARSE_POINT.
_tprintf(_T("Creating reparse\n"));
if (!DeviceIoControl(hFile, FSCTL_SET_REPARSE_POINT, rd, rd->ReparseDataLength + REPARSE_GUID_DATA_BUFFER_HEADER_SIZE, NULL, 0, &dwLen, NULL))
{
CloseHandle(hFile);
DeleteFile(name);
_tprintf(_T("Failed to create reparse\n"));
return -1;
}
CloseHandle(hFile);
Full source code of this app could be found here.
After build and run we get the file. Utility fsutil could help to look into the file we created and make sure that our data is in place.
Processing the RP
It's time to look at this file from the kernel side. I will not go into the details of the mini-filter driver development. There is a good explanation in the official documentation from Microsoft with code examples. Instead we'll take a look at the post callback method.
We need to re-request IRP with the FILE_OPEN_REPARSE_POINT parameter. To do this, we'll call FltReissueSynchronousIo. This function will repeat the request, but with updated Create.Options field.
Inside the structure PFLT_CALLBACK_DATA there is a TagData. If we call FltFsControlFile with FSCTL_GET_REPARSE_POINT parameter, we will get our buffer with data.
// todo we need to check is this actually our tag
if (Data->TagData != NULL)
{
if ((Data->Iopb->Parameters.Create.Options & FILE_OPEN_REPARSE_POINT) != FILE_OPEN_REPARSE_POINT)
{
Data->Iopb->Parameters.Create.Options |= FILE_OPEN_REPARSE_POINT;
FltSetCallbackDataDirty(Data);
FltReissueSynchronousIo(FltObjects->Instance, Data);
}
status = FltFsControlFile(
FltObjects->Instance,
FltObjects->FileObject,
FSCTL_GET_REPARSE_POINT,
NULL,
0,
reparseData,
reparseDataLength,
NULL
);
}
Then we can use this data depending on the task. We can re-request the IRP. Or initiate a completely new request. For example, the LazyCopy project stores the path to the original file in the RP data. The author does not start copying when the file is opened, but only re-saves the data from RP into the stream context of this file. Data starts to be copied the moment the file is read or written. Here are the highlights of his project:
// Operations.c - PostCreateOperationCallback
NT_IF_FAIL_LEAVE(LcGetReparsePointData(FltObjects, &fileSize, &remotePath, &useCustomHandler));
NT_IF_FAIL_LEAVE(LcFindOrCreateStreamContext(Data, TRUE, &fileSize, &remotePath, useCustomHandler, &streamContext, &contextCreated));
// Operations.c - PreReadWriteOperationCallback
status = LcGetStreamContext(Data, &context);
NT_IF_FAIL_LEAVE(LcGetFileLock(&nameInfo->Name, &fileLockEvent));
NT_IF_FAIL_LEAVE(LcFetchRemoteFile(FltObjects, &context->RemoteFilePath, &nameInfo->Name, context->UseCustomHandler, &bytesFetched));
NT_IF_FAIL_LEAVE(LcUntagFile(FltObjects, &nameInfo->Name));
NT_IF_FAIL_LEAVE(FltDeleteStreamContext(FltObjects->Instance, FltObjects->FileObject, NULL));
RPs have a wide range of uses and open up many possibilities for solving various problems. We will analyze one of them by solving the following task.
The task
The game Half-life can run in two modes: software mode and hardware mode, which differ in the way graphics are rendered in the game. After digging in a little with IDA Pro, we can see that the modes differ by the loaded library sw.dll or hw.dll using method LoadLibrary.
Depending on the input arguments (for example, “-soft”), this or that string is selected and passed to the function call LoadLibrary.
The goal of the task is to load the game only in hardware mode without the user noticing it. Ideally, so that the user does not even realize that regardless of his choice, that the game is loaded in hardware mode.
Of course, it would be possible to patch the executable file or replace the dll file, or even just copy hw.dll and rename the copy to sw.dll, but we are not looking for easy ways. Plus, if the game is updated or reinstalled, the effect will disappear.
The solution
I suggest the following solution: write a small mini-filter driver. It will run continuously and will not be affected by reinstalling and updating the game. Let's register the driver for the IRP_MJ_CREATE operation, because every time the executable calls LoadLibrary, it essentially opens the library file. As soon as we notice that the game process is trying to open the sw.dll library, we will return the STATUS_REPARSE status and ask to repeat the request, but this time to open hw.dll. Result: the library we need was opened, even though the user-space asked for another one.
First of all, we need to understand what process is trying to open the library, because we need to turn our trick only for the game process. To do this, right in DriverEntry we will need to call PsSetCreateProcessNotifyRoutine and register a method that will be called every time a new process appears in the system.
NT_IF_FAIL_LEAVE(PsSetCreateProcessNotifyRoutine(IMCreateProcessNotifyRoutine, FALSE));
In this method, we must get the name of the executable file. We could use the function ZwQueryInformationProcess.
NT_IF_FAIL_LEAVE(PsLookupProcessByProcessId(ProcessId, &eProcess));
NT_IF_FAIL_LEAVE(ObOpenObjectByPointer(eProcess, OBJ_KERNEL_HANDLE, NULL, 0, 0, KernelMode, &hProcess));
NT_IF_FAIL_LEAVE(ZwQueryInformationProcess(hProcess,
ProcessImageFileName,
buffer,
returnedLength,
&returnedLength));
If the name matches the target name, in our case it is hl.exe, we have to save its PID.
target = &Globals.TargetProcessInfo[i];
if (RtlCompareUnicodeString(&processNameInfo->Name, &target->TargetName, TRUE) == 0)
{
target->NameInfo = processNameInfo;
target->isActive = TRUE;
target->ProcessId = ProcessId;
LOG(("[IM] Found process creation: %wZ\n", &processNameInfo->Name));
}
So, now we have the PID of the process of our game saved in the global object. We can move to pre create callback. There we should get the name of the file that is trying to be opened. FltGetFileNameInformation will help us with this. This function cannot be called at the DPC interrupt level (read more in about IRQL), however, we are going to make a call at pre create, which guarantees us a level not higher than APC.
status = FltGetFileNameInformation(Data, FLT_FILE_NAME_OPENED | FLT_FILE_NAME_QUERY_FILESYSTEM_ONLY | FLT_FILE_NAME_ALLOW_QUERY_ON_REPARSE, &fileNameInfo);
Then, if our name is sw.dll, then we need to replace it in FileObject with hw.dll. And return the status STATUS_REPARSE.
// may be it is sw
if (RtlCompareUnicodeString(&FileNameInfo->Name, &strSw, TRUE) == 0)
{
// concat
NT_IF_FAIL_LEAVE(IMConcatStrings(&replacement, &FileNameInfo->ParentDir, &strHw));
// then need to change
NT_IF_FAIL_LEAVE(IoReplaceFileObjectName(FileObject, replacement.Buffer, replacement.Length));
}
Data->IoStatus.Status = STATUS_REPARSE;
Data->IoStatus.Information = IO_REPARSE;
return FLT_PREOP_COMPLETE;
Of course, the implementation of the project as a whole is a bit more complex, but I tried to reveal the main points. The whole project with details is here.
Testing the solution
To simplify our test runs, instead of a game, we will run a small application and libraries with the following content:
// testapp.exe
#include "TestHeader.h"
int main()
{
TestFunction();
return 0;
}
// testdll0.dll
#include "../include/TestHeader.h"
#include <iostream>
// This is an example of an exported function.
int TestFunction()
{
std::cout << "hello from test dll 0" << std::endl;
return 0;
}
// testdll1.dll
#include "../include/TestHeader.h"
#include <iostream>
// This is an example of an exported function.
int TestFunction()
{
std::cout << "hello from test dll 1" << std::endl;
return 0;
}
Let's build testapp.exe and link with testdll0.dll, copy them to the virtual machine, and also prepare testdll1.dll. The task of our driver will be to replace testdll0 with testdll1. We will understand that we have succeeded if we see the message “hello from test dll 1” in the console instead of “hello from test dll 0”. Let's run it without a driver to make sure our test application works correctly:
Now let's install and run the driver:
Running the same application, we will get a completely different output to the console since another library was loaded:
Plus, in the application written for the driver, we see logs that say that we really caught our open request and replaced one file with another. The test was successful, it's time to test the solution on the game itself.
I hope it was helpful and interesting.