Do I need to report Illinois, USA state tax while filing tax returns if I was working from home in Indiana for a Illinois based company?

Look at your W-2. Your company should have already withheld state taxes for the correct state(s). Likely that is Indiana, where you both live and work.

I am not a lawyer nor a tax preparer, so this is 100% amateur.

It would appear that Illinois and Indiana do not have a state income reciprocity agreement in place. It is possible that your income has IL state income taxes withheld from it, but since you are a resident of IN, that is the state that has claim of your state-level income tax. Your W-2 should be very clear about what state they paid your state income taxes to, and if that happens to be IL, you will probably end up filing 1 federal form and 2 state forms: a form to IL to get all the taxes withheld refunded to you and a from to IN to pay all the taxes you owed.

Furthermore, you will probably need to look into making estimated state income taxes to IN going forward as they expect you to pay-as-you-go instead of getting one large check when filed. This will stink for you as both IL and IN will have your money and you will only get IL's back as one large check when you file their return. The gov't insists on you paying in as you go, but won't refund as they go. You may be able to talk to your HR dept and have no state tax withheld on your paychecks, though.

But first step in all this is looking on your W-2, Box 15. That will tell you what state got the amount in Box 17 (state income tax).