If you use Python a lot then you probably know that many people claim that Python takes up more time to execute. Well, you probably have seen approaches like the total time spent to execute a part of code or something like that but sometimes you need something more than that. What about RAM usage? Nobody really talks about that but it’s equally essential. This might look really insignificant but it’s actually pretty important while writing code for production.
We will be using memory-profiler from PyPi. We will also be needing requests to test the functionality. To do so, simply type the following in your terminal
pip3 install memory-profiler requests
Note: If you are working on windows or using a virtual env, then it will be pip instead of pip3
Now that everything is set up, rest is pretty easy and interesting obviously. Create a new file with the name word_extractor.py and add the code to it. Below is the implementation of the code. Everything is pretty well documented as in-line comments.
Python3
# importsfrom memory_profiler import profileimport requests class BaseExtractor: # decorator which specifies which # function to monitor @profile def parse_list(self, array): # create a file object f = open('words.txt', 'w') for word in array: # writing words to file f.writelines(word) # decorator which specifies which # function to monitor @profile def parse_url(self, url): # fetches the response response = requests.get(url).text with open('url.txt', 'w') as f: # writing response to file f.writelines(response) |
Notice the @profile this is a decorator. Any function which is decorated by this decorator, that function will be tracked. Now, our main code is ready. Let’s write the driver code which will call this class functions. Now, create another file called run.py and insert the following code in it.
Python3
from word_extractor import BaseExtractor if __name__ == "__main__": # url for word list (huge) # word list in array array = ['one', 'two', 'three', 'four', 'five'] # initializing BaseExtractor object extractor = BaseExtractor() # calling parse_url function extractor.parse_url(url) # calling pasrse_list function extractor.parse_list(array) |
So, basically now we are done. You will notice that parse_url() will consume more memory than parse_list() which is obvious because parse_url calls a URL and writes the response content to a text file. If you open the link, then you will find that the word list is huge. So, now to test your code, simply run the run.py file. You can do so by typing
python3 run.py
Note: If you are working on windows or using a virtual env, then it will be python instead of python3
If everything ran successfully, then you should see something like this
Memory Profiler Stats
An important thing to remember is that memory-profiler itself consumes a significant amount of memory. Use this only in development but avoid it in production.
