Enumerating Job Objects

June 17, 2017

no comments

Job objects have been around since Windows 2000, providing a convenient way to set limits and otherwise manage a set of processes. Up until Windows 8 job objects were used sparingly, because a process could only be associated with a single job at most. That would mean an application wanting to set some limits on a process it does not create explicitly had no way of knowing whether that process was already part of a job. If it were, assigning it to another job would simply fail.

Starting with Windows 8, jobs can be nested, effectively creating a job hierarchy when processes are added to multiple jobs. The general rule is that a limit set by a parent process cannot be loosened by a child process (but it can be made more limiting).

Today, jobs are used extensively. For example, all UWP/Store/Modern processes are managed with a job. Any background tasks created for that process are managed under the same job. Converted Win32 applications using the Desktop Bridge ("Centennial") run in a job as well, that is promoted into a Silo – the internal name for a Windows Container. Docker and Docker-like platforms use Silos as well (called server silos).

One oddity of jobs is the lack of documented APIs to enumerate them. If we look at processes, there are several ways to enumerate these: The EnumProcesses function, which is the simplest, returning just a list of process IDs. More options are available with WTSEnumerateProcesses and CreateToolhelp32Snapshot. See my post here for more on enumerating processes.

There is no EnumJobs or something similar. Within the kernel, all jobs are stored as a system-wide linked list pointed to by the PspJobList global variable (much like the global process linked list is stored in the PsActiveProcessHead variable).

Using the PspJobList directly from a driver has a couple problems: first, the symbol is not exported, so how can the driver locate it? Second, enumerating is tricky and requires acquiring correctly a lock that protects that list (jobs may be added or removed while the enumeration takes place), so which lock is it, and where is it located?

Using a kernel debugger, we can issue the (currently) undocumented command, !joblist to get a flat list of all jobs in the system in question:

lkd> !joblist
Job at ffffb487f6f97060
  Basic Accounting Information
    TotalUserTime:             0x0
    TotalKernelTime:           0x0
    TotalCycleTime:            0x0
    ThisPeriodTotalUserTime:   0x0
    ThisPeriodTotalKernelTime: 0x0
    TotalPageFaultCount:       0x0
    TotalProcesses:            0x1
    ActiveProcesses:           0x1
    FreezeCount:               0
    BackgroundCount:           0
    TotalTerminatedProcesses:  0x0
    PeakJobMemoryUsed:         0x264
    PeakProcessMemoryUsed:     0x264
  Job Flags
    [wake notification allocated]
    [wake notification enabled]
    [timers virtualized]
  Limit Information (LimitFlags: 0x3800)
  Limit Information (EffectiveLimitFlags: 0x3800)
    JOB_OBJECT_LIMIT_BREAKAWAY_OK
    JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
    JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK
Job at ffffb487f6f9c060
  Basic Accounting Information
    TotalUserTime:             0x0
    TotalKernelTime:           0x0
    TotalCycleTime:            0x0
    ThisPeriodTotalUserTime:   0x0
    ThisPeriodTotalKernelTime: 0x0
    TotalPageFaultCount:       0x0
    TotalProcesses:            0x1
    ActiveProcesses:           0x1
    FreezeCount:               0
    BackgroundCount:           0
    TotalTerminatedProcesses:  0x0
    PeakJobMemoryUsed:         0x264
    PeakProcessMemoryUsed:     0x264
  Job Flags
    [wake notification enabled]
    [timers virtualized]
  Limit Information (LimitFlags: 0x1800)
  Limit Information (EffectiveLimitFlags: 0x3800)
    JOB_OBJECT_LIMIT_BREAKAWAY_OK
    JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK
Job at ffffb487f6f96060
...

Clearly, the debugger knows how to obtain a list of jobs. Can we do the same?


Enumerating manually with the global variable and using the correct lock is too risky. Fortunately, there is an alternative: The PspGetNextJob function. As the name suggests, this function can be used to enumerate job objects safely, starting with a NULL job, which returns the first job, and subsequently using the current job as input to get the next one, etc.:

for (PEJOB job = PspGetNextJob(nullptr); job; job = PspGetNextJob(job)) {
    // do something with job
}

This looks easy enough. However, the PspGetNextJob function is not exported, so how can we locate it?

Although the function is not exported, it is available with the public symbols. If we could get the offset of the function using the symbols, we could easily calculate the address of the function and use it in a simple software driver to get back the information. This is exactly what I did.

Working with symbols can be done using the DbgHelp API. Some of its functions are fairly straightforward, while others require more complex coding. For the purpose of getting a global symbol’s address, it is the former.

The order of calls to get a symbol’s address is the following:

  1. SymSetOptions is optionally called to set global DbgHelp options.
  2. SymInitialize is called to initialize DbgHelp and optionally set a symbol search path and a process to invade (none in our case).
  3. SymLoadModuleEx is called to load symbols for a module (in our case, its the kernel itself, ntoskrnl.exe).
  4. Finally, a call to ‘SymFromName’ provides the necessary information, returned as a SYMBOL_INFO structure which holds all available information on a particular symbol.

The address stored in that SYMBOL_INFO is actually a Relative Virtual Address (RVA), which is a simply an address relative to the base address of the module (obtained from SymLoadModuleEx). So we need to subtract the RVA from the module address to get the actual offset from the start address of the kernel. But how can we get that?

The PSAPI API provides the `EnumDeviceDrivers’ function, which returns an array of base addresses for all kernel modules, starting with the kernel itself, followed by the HAl, followed by the rest of the drivers in the kernel.
This means we can use the following function that returns the kernel base address:

void* GetKernelBaseAddress() {
    void* kernel;
    DWORD needed;
    if(EnumDeviceDrivers(&kernel, sizeof(kernel), &needed))
        return kernel;
    return nullptr;
}

Given all the above, we can use the following code to get the address of PspGetNextJob (the SymbolHandler class simply wraps the DbgHelp API calls mentioned earlier):

SymbolsHandler handler;
auto address = handler.LoadSymbolsForModule("%systemroot%\\system32\\ntoskrnl.exe");
auto PspGetNextJobSymbol = handler.GetSymbolFromName("PspGetNextJob");
if (PspGetNextJobSymbol == nullptr)
    return Error("No symbols have been found or SymSrv.dll missing." 
    " Please check _NT_SYMBOL_PATH environment variable");

// calculate the exact function address

void* PspGetNextJob = (void*)((ULONG_PTR)KExploreHelper::GetKernelBaseAddress() 
    + PspGetNextJobSymbol->GetSymbolInfo()->Address - address);

Next, we need to pass this discovered address to our driver for enumeration purposes. The driver I’ve written accepts this information using a DeviceIoControl call:

DWORD returned;
void* jobs[1024];
if (::DeviceIoControl(hDevice, KEXPLORE_IOCTL_ENUM_JOBS, &PspGetNextJob, sizeof(PspGetNextJob), 
    jobs, sizeof(jobs), &returned, nullptr)) {
    int countJobs = returned / sizeof(jobs[0]);

We haven’t seen the driver code yet, but the returned values in the array are addresses of job objects. Now what can we do with that in user mode? Not much, really.

Thankfully, we can use our driver once again and ask it to open a handle to each job object. Then we can use the QueryInformationJobObject to obtain a lot of information on the job in question. Currently, the tool just gets the name of the job object (if any) and the processes currently active within a job.

Well, actually getting the name of the object is non-trivial, as there is no documented way to do that. I used the semi-documented NtQueryObject/ZwQueryObject function to get then name of the object:

auto bstr = std::make_unique<BYTE[]>(512);
auto str = reinterpret_cast<UNICODE_STRING*>(bstr.get());  
NtQueryObject(hJob, (OBJECT_INFORMATION_CLASS) ObjectNameInformation, str, 512, nullptr);

Lastly, how can our driver open a handle to the job object?
The kernel API provides the documented function ObOpenObjectByPointer that allows getting a handle to any object using its address. We can request that handle not be a kernel handle so that it can be returned to user mode.

Here’s an example output of the JobList tool with the -named argument that only shows named jobs:

C:\Tools>JobList.exe -named
JobList v0.1 (C)2017 by Pavel Yosifovich

Found 485 jobs.
Job address         Name                                                        Processes: total/active/terminated (list)
-------------------------------------------------------------------------------------------------------------------------

0xFFFFB487F59A7890 \BaseNamedObjects\WmiProviderSubSystemHostJob                           86/2/0 ( 6952 15408 )
0xFFFFB487F59A6190 \BaseNamedObjects\WmiProviderSubSystemSpecialHostJob                    0/0/0 ( )
0xFFFFB4880238A6F0 \Sessions\1\BaseNamedObjects\7573f6ec-d89b-4bce-b8a7-39ea918ef376       63/1/0 ( 14596 )

Conclusion

The combination of a driver and kernel symbols opens up many possibilities for kernel exploration. That’s why my repo is called KernelExplorer; it’s not that yet, but I plan to create a nice GUI to shows various parts of the system, such as job hierarchies.

As another example, we can dig into the EJOB structure using offsets obtained from symbol information (structures like EJOB and EPROCESS are undocumented and subject to change between Windows releases).

The JobList tool executable can be found here. The driver is bundled inside by packaging it (and DbgHelp.dll and SymSrv.dll) inside resources as I described in principle in this blog post. The tool is currently x64 only (just laziness on my part).

The complete source code is on Github here.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*