Thursday, January 28, 2021

Process Memory Utilization / Iron Scripter Challlenge (20 Jul 2021)

I've heard of the Iron Scripter Challenge, but never really looked into it. This week I decided to take the plunge and I'm glad I did. Basically, the Iron Scripter Challenge is a series (I believe monthly) of scripting challenges in PowerShell. If your like me and you love challenges, or learn by doing, I highly recommend it. Here is a link to the challenge I worked on this week https://ironscripter.us/a-memory-reporting-challenge/ 

The overall goal was to collect memory data on running processes by process name. For example, if there are 20 instances of Chrome process running, combine the memory data for all of them. I modified this requirement a little and included the path with the name because I know malicious process can use legitimate names running in incorrect locations. For example, what if one of those 20 Chrome processes was executed from C:\Windows\Temp? My Spidey sense would be tingling. 

Here's the basic data I needed to collect: 
  1. Process name  
  2. Total number of processes 
  3. Total WorkingSet 
  4. Percentage of in-use memory

While it would be easy to write this in a verbose script with variables and foreach loops to solve this, I thought a good challenge for me was to write this in one command with many pipes. Additionally, I wanted to make this as efficient as possible with that limitation.

Process objects returned from the Get-Process, Group-Object, and Measure-Command cmdlets provide all the data we need. I'll break down each here with the corresponding requirement:
    1. Process name  (Get-Process ProcessName property)
    2. Total number of processes (Use Group-Object Count property)
    3. Total WorkingSet (Use Measure-Object Sum property on the WorkingSet of the group)
    4. Percentage of in-use memory (Divide the WorkingSet for the group by the WorkingSet sum for all properties)
1   Get-Process |
2       Group-Object Name, path |
3           Select-Object @{
4                   n="ProcName"
5                   e={($_.name -split ", ")[0]}
6               },
7               @{
8                   n="Path"
9                   e={($_.name -split ", ")[-1]}
10              },
11              @{
12                  n="TotalProcs"
13                  e={$_.Count}
14              }, 
15              @{
16                  n="WorkingSetTotal"
17                  e={($_.Group.WorkingSet | Measure-Object -Sum).sum}
18              },
19              @{
20                  n="Percentage"
21                  e={($_.Group.WorkingSet | Measure-Object -Sum).sum / 
22                  ((Get-Process).workingset | Measure-Object -Sum).Sum 
23                  }
24              }

I used calculated properties for most of the requirements. The Group-Object command will give me a Name, Count, and Group property.  Name will be a string in the format of "Name, Path". Depending on the user's permissions when running the command, some process paths may be null. By using the -split operator with the regular expression of ", " I account for these instances. In those cases the Path value will be the same as the name because in an array with 1 element, item [0] and [-1] are the same. The Count property will be the total number of processes for that group.  The group property will be all the process objects belonging to that group, so $_.group will be an array of those objects.

After this first go around, I wasn't quite satisfied with the Percentage calculation even though it worked just fine. I didn't like the fact I was running Get-Process again for each group when I already ran it. This is where Tee-Object comes. By using Tee-Object, I can assign the original process objects collected in the first command on the pipeline to a variable then call it later down the pipe. Here's the updated version.

1   Get-Process |
2       Tee-Object -Variable Procs |
3           Group-Object Name, path |
4               Select-Object @{
5                  n="ProcName"
6                  e={($_.name -split ", ")[0]}
7               },
8               @{
9                  n="Path"
10                 e={($_.name -split ", ")[-1]}
11              },
12              @{
13                 n="TotalProcs"
14                 e={$_.Count}
15              }, 
16              @{
17                 n="WorkingSetTotal"
18                 e={($_.Group.WorkingSet | Measure-Object -Sum).sum}
19              },
20              @{
21                 n="Percentage"
22                 e={($_.Group.WorkingSet | Measure-Object -Sum).sum / 
23                    ($Procs.workingset | Measure-Object -Sum).Sum 
24                    }
25              }


Being satisfied with this, I moved on to the "bonus" requirements. Here they are:
    1. Create a PowerShell function around your code
    2. Support querying a remote computer
    3. Provide sorted results
    4. Provide formatted output
    5. Services run in a process so create a command to report on the same memory usage for a given service. It is OK if the service shares a process. Although you might want to indicate that.
I sat out to tackle the first four.  For the first requirement, I decided to create 2 functions a "Get" and a "Show".  I did this because a couple requirements were formatting and sorting which are not appropriate for a "Get" function, so I split them up into 2. Both functions will work the same way in fact I just call the "Get" function from the "Show" function to get the data before formatting.

Let's break down the "Get" function first.  Here is the top of the function with the param block.

1   function Get-ProcessMemory {
2       [CmdletBinding(DefaultParameterSetName = 'None')]
3       [OutputType([PSCustomObject])]
4   
5       param (
6           # Remote computer or collection of remote computers
7           [Parameter(Mandatory = $trueValueFromPipeline = $trueParameterSetName = 'CN')]
8           [string[]]
9           $ComputerName,
10  
11          # PSCredential for remote computer(s)
12          [Parameter(ParameterSetName = 'CN')]
13          [pscredential]
14          $Credential,
15  
16          # PSSession for remote connection
17          [Parameter(Mandatory = $trueParameterSetName = 'Session')]
18          [System.Management.Automation.Runspaces.PSSession]
19          $Session
20      )
21         
22  

I intentionally used parameter names that match some of the Invoke-Command parameters. I did this because I intend to use that cmdlet to satisfy the second bonus requirement using the $PSBoundParameters automatic variable. I also have 3 parameter sets, even though you only see 2 explicitly named in the parameters. The third one is default "None" which will happen when no parameters passed (technically it would be the default if the parameter set couldn't be determined). What I'm trying to allow here is the ability to just call the function on the local machine without passing any arguments, but include them for remoting if needed.

On to the process block:

23      process {
24          $Command = @'
25              Get-Process |
26                  Tee-Object -Variable Procs |
27                      Group-Object Name, path |
28                          Select-Object @{
29                              n="ProcName"
30                              e={($_.name -split ", ")[0]}
31                          },
32                          @{
33                              n="Path"
34                              e={($_.name -split ", ")[-1]}
35                          },
36                          @{
37                              n="TotalProcs"
38                              e={$_.Count}
39                          }, 
40                          @{
41                              n="WorkingSetTotal"
42                              e={($_.Group.WorkingSet | Measure-Object -Sum).sum}
43                          },
44                          @{
45                              n="Percentage"
46                              e={($_.Group.WorkingSet | Measure-Object -Sum).sum / 
47                                  ($Procs.workingset | Measure-Object -Sum).Sum 
48                              }
49                          }
50  '@ #Command here-string to invoke
51  
52  

I created a Here-String containing my original command. I did this so I can use it for both Invoke-Expression for the local machine or Invoke-Command for remote machines.  On to the next/last portion.

53          if ($PSCmdlet.ParameterSetName -eq "None") {
54              Invoke-Expression -Command $Command
55          } #if ParameterSetName is None (local machine)
56          
57          else {
58              $InvokeCommandArgs = $PSBoundParameters #works because I use the same parameter names
59              $InvokeCommandArgs.ScriptBlock = [scriptblock]::Create($Command)
60              Invoke-Command @InvokeCommandArgs
61          } #else - ParameterSetName is NOT None (remoting)   
62  
63      } #Process Script Block for Get-ProcessMemory Function 
64  
65  #Get-ProcessMemory Function Definition
66  


In this last section I determine if it is going to be ran on the local machine (ParameterSet -eq "None") or remote machine. For local machine I just used the Invoke-Expression cmdlet to run my command. I could have converted it to a script block and called the Invoke method, but I like to use native cmdlets when I can.  

For remote machines, I take advantage of using standard names for my parameters so I can just use the $PSBoundParameters automatic variable to splat Invoke-Command.  I did convert $Command to a scriptblock to make this call.

The "Show" function has the exact same parameters so I'll skip that and go straight to the process block.

100     process {
101         
102         Get-ProcessMemory @PSBoundParameters |
103             Sort-Object WorkingSetTotal -Descending |
104                 Format-Table -Property @{
105                     n="Name"
106                     e={$_.ProcName}
107                     w=35
108                 },
109                 @{
110                     n="Path"
111                     e={$_.Path}
112                     w=30
113                 },
114                 @{
115                     n="Procesess"
116                     e={$_.TotalProcs}
117                     w=10
118                 },
119                 @{
120                     n="Usage/MB"
121                     e={$_.WorkingSetTotal / 1MB}
122                     f="N2"
123                     w=15
124                 },
125                 @{
126                     n="Percentage"
127                     e={$_.Percentage}
128                     f="P2"
129                     w=15
130                 }
131         
132     } #Process Script Block for Show-ProcessMemory Function
133 

Since the parameters were the same I just called my "Get" function splatting the $PSBoundParameters variable. I decided to sort it by memory usage so the highest usage is at the top. Then, I put all the data neatly in a table using format specifiers for the numbers and specified the width as well. If you are unfamiliar with this take a look at about_calculated_properties. This is about the only place I use shorthand. "n" is short for "name", "e" is short for "expression", "f" is short for "formatstring" and "w" is short for "width".

Finally I bundled it up in a module, exporting both functions. All the code is available for download in my github repository here https://github.com/ralphmwr/MemoryUse

Here is a screen shot of the Show-ProcessMemory function in action:


Looking forward to the next Iron Scripter Challenge.

Please comment if you have questions or suggestions.