No, you should not be terribly worried, unless you did horrible no-nos like putting credentials in your source tree. So the party who accessed these files saw part of your source history. Big deal. That gives them a window into understanding how your application does things internally, and might assist them in searching for vulnerabilities, but it's nothing catastrophic.
It would be a very good idea to do a security review on your application, especially the parts of the code you suspect they accessed, in case there are obvious bugs they can exploit. But you don't need to panic.
