PHP has two functions which will return amount of memory allocated by the process: memory_get_usage and memory_get_peak_usage.

int memory_get_usage ([ bool $real_usage = false ] )

Returns current amount of the memory in bytes. If $real_usage is false it will return only used amount of memory. If set to true it will return all memory allocated from the system.

int memory_get_peak_usage ([ bool $real_usage = false ] )

Returns maximum amount of the memory in bytes from the start of the process till the moment it was called. If $real_usage is false it will return only used amount of memory. If set to true it will return all memory allocated from the system.

Problem

In most cases these to functions return usefully data. But they don’t  return the amount of the memory which is used by the PHP resources.

Quick reminder, list of PHP types:

• boolean
• integer
• float floating-point number
• string
• array
• object
• callable
• iterable
• resource
• NULL

Resource is a variable that holds the reference to the external resource. Memory for resource is allocated outside of the ZEND engine in the external libraries. So memory_get_usage and memory_get_peak_usage will not return that memory, although your PHP script is using that memory, and that memory was in a fact allocated from the system.

In order to be sure how efficient you script is, you need to be aware which resources you are using and how much burden they put on the whole system.

For list of functions that create or destroy resources please check: php.net/manual/en/resource.php.

Typical example of resource type usage, which can consume huge amounts of memory is XML parsing. SimpleXMLElement as returned by the, for example, simplexml_load_file is in a fact a resource.
PHP programmers works with SimpleXMLElement as it is “ordinary” object, but since it is not an object you can not store it in $_SESSION and its node values must be cast to some other type. And of course it’s memory usage is not reported by PHP memory_get_* functions.

Solution

If you are using Unix type OS, you can check process memory consumption of the process in the /proc virtual file system. On my Linux system it is in:

Example

example_01.php:

Output:

From above output we are interested in VmRSS which stands for Virtual Memory Resident Set Size. Resident Set Size is allocated memory  by the process which resides in the main memory (RAM). Rest is in the swap space. In the above example VmSwap is 0 kb.

Lets write function that will return VmRss+VmSwap:

example_02.php

Output:

Excellent, now it is time for realistic test and comparison between PHP memory_get_peak_usage and memory_get_usage and our  get_process_memory_usage .

Test

Parse small and then large xml file and analyze memory consumption as reported by three function mentioned above. We will use fragment of XML feed from one of the bicycle shop partners from my bicycle shopping directory www.bicycle-discounts.com.

Small XML file: feed_sml.xml.gz . Around 13 items , uncompressed size on disk 29Kb.

Large XML file: feed_big.xml.gz . Around 40000 items, uncompressed size on disk 109Mb.

Note: ungzip them before running the example.

test.php

Test 1:

Parse small xml file:

Test 2:

Parse large xml file:

Comparison

function	small xml (kb)	big xml (kb)
memory_get_usage()/1024	352.25	352.25
memory_get_usage(true)/1024	2048	2048
memory_get_peak_usage()/1024	391.54	391.54
memory_get_peak_usage(true)/1024	2048	2048
memory_get_process_usage()	13688	478096

As we can see, memory usage as reported by PHP memory_get_* functions is the same when processing small xml file and big xml file. But there is a huge difference in real memory consumption which is reported by our custom memory_get_process_usage which relays on /proc/<PID>/status. 13688kb vs 478096kb!

This demonstrate how PHP memory_get_* function are not good choice when analyzing memory consumption of PHP scripts that are using resources. Also if we were to write PHP script which processes big XML files, using simplexml_* functions such as simplexml_load_file or even worse simplexml_load_string are not good choices. Since they read the whole XML structure into the memory, instead of reading it bit by bit.

In the next post I will show you how to write memory efficient PHP script which processes  big XML files, while not sacrificing conveniences provided by simplexml_* functions. Stay tuned. 🙂

References

• php.net/manual/en/language.types.resource.php
• php.net/manual/en/resource.php
• en.wikipedia.org/wiki/Zend_Engine
• en.wikipedia.org/wiki/Procfs
• en.wikipedia.org/wiki/Resident_set_size