How can I plot more 10k points using matplotlib?

Question

I'm trying to plot spectrum contains ~11 000 points, so my program hangs (past my code line), below my code. How can I fix it?
import matplotlib.pyplot as plt

try:
    spectrum_file = open('external_data/spectrum.txt', 'r')
    spectrum_data = spectrum_file.read()
    print('Spectrum has imported successfully')
except:
    print('Error of importing spectrum')

try:
    spectrum_data = spectrum_data.split('n')
    x = [row.split('t')[0] for row in spectrum_data] # ['1', '2', ... , '11 000']
    y = [row.split('t')[1] for row in spectrum_data]
    print('Spectrum data has separated successfully')
except:
    print('Error of separating the spectrum data')

plt.plot(x, y, label='Spectrum')
plt.show()

JohanC · Answer

Before considering that matplotlib wouldn't be able to plot more than 10,000 points, you really should check your code. In fact, matplotlib can plot millions of points without problems. You could check your code by limiting the plot to the first 100 or so points and note strange behavior.
The main problem in the code is that strings are read into x and y arrays. Matplotlib tries to plot them as if they were names, not numbers.  With 10,000 points the code is extremely slow in trying to cope with too many strings.
The solution is to convert the strings:
x = [float(row.split('t')[0]) for row in spectrum_data]
y = [float(row.split('t')[1]) for row in spectrum_data]

Instead of reading the file "by hand", it is usually recommended to use specialized functions, such as read_csv from pandas. Or loadtxt from numpy. Such functions automatically convert the fields while looping through the file.
xy = np.loadtxt('external_data/spectrum.txt')
x = xy[:,0]
y = xy[:,1]

PS: Here is an example of matplotlib with 10 million points:
import matplotlib.pyplot as plt
import numpy as np

x = np.random.normal(0, 1, 10000000)
y = np.random.normal(0, 1, 10000000)
plt.plot(x, y, ls='', marker=',', markeredgecolor='none')

plt.show()

How can I plot more 10k points using matplotlib?

One Answer

Add your own answers!

Ask a Question