"No, you cannot work as freelancer if u have green card.
I have the same problem."
100% "no" if it is mentioned in your residence permit that you can not work as a freelancer. otherwise it may be possible to start and win a legal dispute, if a local employment office makes a mistake and arguments a refusal wrongly (hint: they should say that they do not have to issue a subsequent work permit accordingly to §6 Abs.3 IT-ArGV at all, therefore due to a situation on a labour market decided not to, then there is no chance; but if they say that GC can not work as a freelancer ...

)
"?Does any one know what are the legal paper work to be done if a Greencard visa holder wants to work as a Freelancer."
As i said it depends whether a work as a freelancer is explicitly prohibited to you by corresponding remarks in your residence permit. if it is (and in most cases it is, excluding i guess only in Rheinland-Pfalz) then you have to change your residence permit firstly on a "business"-type residence permit. this will involve a long process of gathering approving papers from local IHK (industry and commerce chamber), from credit institutes, etc., and will result in your inability to work as a normal employee once your residence permit is changed.