An autoencoder learns to copy its input through a narrow middle. Trained only on normal SSH messages, it learns their patterns well enough to rebuild them, and it can't rebuild what it has never seen. Lines it rebuilds badly are anomalies. This page follows one line of a real auth.log through every step: tokens (characters, words or subwords, your choice), an embedding, Conv1D layers that squeeze the line down and expand it again, and the reconstruction error that decides. Each stage has the matching TensorFlow/Keras code, with the output it prints; open it with Show code.
Every number below has one of four origins. Each stage is tagged with the same labels.
auth.log, attacks included.Here: the sshd messages, turned into token ids.The data is the auth.log from Elastic's suspicious login activity example (Apache-2.0): one internet-facing Ubuntu server, 27 March to 20 April. Every line starts with a date, the host name and the program that wrote it.
Several programs write to this file. Only sshd, the SSH server, is about SSH connections, so we keep its lines. We also drop the start of each line (date, host, process id, shown faint above): it changes on every line and says nothing about what the message means. What's left is the message.
The file has no labels. To train on normal lines and to check the results, each message gets a label from its type with a few simple rules. The model never sees these labels.
The data is unbalanced, the "wrong" way round. Most sshd lines are attacks: bots trying passwords all day long. An autoencoder learns whatever is common, so trained on everything it would learn that brute-force attempts are normal, and stop flagging them. So it's trained on legitimate lines only (semi-supervised): you need a period, or a filter, you trust to be clean. In practice that's the hardest part.
The validation lines are never trained on; their scores set the threshold in step 9. The test lines, legitimate and attack, are used only to measure the result.
The model can't read text: the message is split into tokens, and every token seen in the training lines gets an id. Anything else becomes id 1, "unknown" (dashed below). Every line becomes exactly the same number of ids: longer messages are cut, shorter ones padded with id 0. How to split is a design choice, and it changes what the model can notice. Pick one in the toolbar; every step below uses its own model, trained with that tokenizer.
Pick a line in the toolbar, edit it, or type your own; click a token to inspect it in the steps below.
An id is just a label: 7 isn't "more" than 6. The embedding layer swaps each id for a learned row of 8 numbers, so tokens that behave alike can end up with similar rows. The whole line becomes a matrix with one column per token below and one row per embedding number. Blue is positive, red negative, and stronger colour means a larger number; padding is grey. Click a column to inspect that token.
A Conv1D filter slides along the line and looks at 5 neighbouring tokens at a time. It multiplies their 5×8 embedding numbers with its own 5×8 learned weights and adds them up: a high value means "the pattern I look for is here", such as a run of digits or the word "port" followed by a number. 16 filters give 16 numbers per position, and ReLU sets negatives to 0. Max pooling then keeps the larger of every two neighbours, halving the length. A second Conv1D and pooling leave the bottleneck.
The decoder only gets the bottleneck. Upsampling repeats every position twice, and a Conv1D after each repeat lets neighbours differ again. The last layer is a Conv1D with filters 1 token wide: it turns each position's 16 numbers into one score per token in the vocabulary, and softmax makes those probabilities. The model never copies the input: it has to rebuild the line from the numbers in the bottleneck.
The reconstruction is the most likely token at each position. The error for one token is −log of the probability the model gave the true token: 0 when it was certain and right, about 16 when it was sure of something else. The line's anomaly score is the average over its tokens. Top row: the line; bottom row: the reconstruction, with mistakes underlined and the background tinted by the error.
The message is empty, so there's nothing to reconstruct. Pick a line in the toolbar or type one.
Training is ordinary supervised learning with a twist: the target is the input itself. Keras shows the model the training lines, measures the error at every real token (padding doesn't count), and nudges all the weights to reduce it, using the Adam optimizer. The validation lines are only measured: when validation loss stays close to training loss, the model has learned general patterns rather than memorizing lines.
Why it can't just learn to copy everything: the bottleneck is too small to store an arbitrary line, so the weights have to spend their capacity on what the training lines have in common: their fixed words, the shape of an IP address, the known users and clients. Each tokenizer has its own model, trained the same way on the same lines; these weights came from the run shown here, rounded to 4 decimals.
Every line gets a score. Above the threshold it's flagged as anomalous. The default threshold is the 99th percentile of the validation scores: about 1 in 100 legitimate lines will be flagged, a false alarm rate chosen up front. Move it and watch the trade-off between missed attacks and false alarms. Scores run from about 0.0001 to 16, so the slider and the chart use a logarithmic scale.
In this log, attack messages use different words than normal ones ("Failed", "Invalid user", "[preauth]"), so they're easy to separate. The harder cases are below: legitimate lines that are simply rare, and attacks that look normal one line at a time.
Each model at its own default threshold. Every one catches all attack lines here, so the differences are in what else gets flagged. Masking numbers mostly stops the rare client from being flagged: good if new clients are routine, bad if a new client is exactly what you want to see. Models that rebuild normal lines almost perfectly (masked characters, words) give tiny scores, so their threshold rests on very small differences between 225 validation lines, and masked characters ends up with more false alarms. None of them sees the brute-force success. Click a row to switch.
Test lines above the threshold, highest score first (identical messages are shown once). The reconstruction drifts towards the normal lines the model knows.
Legitimate test lines, never seen in training. The fixed words come back exactly; what changes is what can't be predicted, mostly port numbers (unless the tokenizer masks them).
The autoencoder flags what's unusual, not what's malicious. A client that logged in only a few times, or a message type that hardly appears in training, scores high.
Attack lines below the threshold, closest to it first.
Server restarts and debug messages: harmless, but never in the training set, so they're flagged too.
24.151.103.17 sent about 150 failed passwords for elastic_user_0, and then logged in successfully. Each of those successful logins is a perfectly normal line, so it scores low. Catching this needs context across lines, such as failures per client in the last hour, which is exactly what Elastic's original example models.
Paste lines from an auth.log (whole lines, or just the messages), or load a sample from this log. Each line goes through the model on its own and gets its own score; the score stays attached to that line. Lines at or below the threshold are removed as normal. What remains are your original lines, unchanged: the reconstruction under each one only shows why it was kept. Nothing is uploaded: it all runs in your browser.
Highest score first. C, M, W and B show which of the four models (characters, masked, words, BPE) flag the line at their own threshold.