Hi Arya!
Lead with friendly honesty and make it about your schedule rather than their behavior. The next time you see them, try something like, "Hey, I love chatting with you, but I'm often in the middle of work or resting. Could you text me before you swing by? I'm usually free after 6 or on Saturday mornings." Offering a couple of clear time windows shows you still want a relationship, just with structure. Keep your tone light and appreciative, then stop talking so they can agree without feeling cornered.
When they do drop by unannounced, keep the interaction short at the door and restate the boundary. Open the door, smile, and say, "I can't talk right now. Text me and we'll set a time later." If it keeps happening, add a calm, matter of fact consequence like, "If I don't get a text first, I probably won't be able to answer." A small physical cue helps too. I put a 3x5 card above my doorbell that says "Working 9 to 5. Please text before knocking" and after a week of repeating the same line once or twice, the habit stuck without any hard feelings.