Make SURE you carefully wipe away such dirt and debris BEFORE unscrewing the cap. Also, once you do remove the cap, clean its underside and O Ring carefully to make sure you are not introducing dirt into the engine when you reinstall and tighten the cap.

I wouldn't be concerned for a couple of reasons.

1. Diesel engines require a break in period and you are just getting started in the break in period of time.
2. The residue collecting the dirt and dust could be from when the engine oil was put into the engine and some was spilled onto the engine and wiped up, but not thoroughly removed the oil residue and dirt will stick to it.

I do want to commend you for even noticing this issue. Many owners never open the hood or check anything so its good you are paying attention to such issues and that you asked about it. Good job on being vigilant and paying attention to your machine.

Here is a screenshot of the cap and it shows the O ring which should be on the cap. Make sure the O ring is on the cap. Also, there are usually at least 2 and sometime 3 of these caps on the engine and occasionally, people will wonder which one they should use to fill the engine oil and they often ask where there are multiple caps like these.

You can use any of the caps to add the engine motor oil, it doesn't matter. Personally, I prefer to use the cap highest on the engine as this means the oil will run down over the valve train parts when added to the highest oil cap. Also, the reason why there are so many different caps is because the engine is used in a variety of applications and some of the caps might not be accessible for use on some of the applications, so several caps are used to make sure its possible to easily add engine oil when necessary.