Question

Counting the non null rows in multiple columns combined in python

Answer 1

When using pandas, try to avoid performing operations in a loop, including apply, map, applymap etc. That's slow!

If you want to count the missing values in each column, try:

df.isnull().sum() or df.isnull().sum(axis=0)

On the other hand, you can count in each row (which is your question) by:

df.isnull().sum(axis=1)

It's roughly 10 times faster than Jan van der Vegt's solution(BTW he counts valid values, rather than missing values):

In [18]: %timeit -n 1000 df.apply(lambda x: x.count(), axis=1) 1000 loops, best of 3: 3.31 ms per loop In [19]: %timeit -n 1000 df.isnull().sum(axis=1) 1000 loops, best of 3: 329 µs per