Toots from 2024-02-06
This newsletter subscription could have been an atom:entry
This newsletter subscription could have been an atom:entry
Mastodon Source 🐘
@missmythreyi Nice - ty! Added to queue. Happy to read a…
@missmythreyi Nice - ty! Added to queue.
Happy to read and provide thoughts whenever you have something to share. Sounds very interesting 👍
Mastodon Source 🐘
The words “wrong" and “careful” are doing a tremendous amount of heavy lifting in this sentence…
The words “wrong" and “careful” are doing a tremendous amount of heavy lifting in this sentence:
“Calibration has so far been studied in non generative (e.g., classification) settings, especially in Software Engineering. However, generated code can quite often be wrong: Developers need to know when they should e.g., directly use, use after careful review, or discard model-generated code.”
https://arxiv.org/abs/2402.02047
Mastodon Source 🐘
Given the oracle is known unreliable, this seems odd.
“We consider…reflective probability, obtained by re-invoking the model, prompting it to estimate how confident it is in the correctness of of the code it had just generated (See Figure 1).”
Mastodon Source 🐘
@missmythreyi @b0rk It is…
@missmythreyi @b0rk It is absolutely perfect 👌 I wish I had learned about it earlier for myself 😀
Cool research. Would you recommend a starting point for me to read more? Things are only retroactively “inevitable” and “easy” after someone has done the hard work to pull certainty out of uncertainty.
Mastodon Source 🐘
It’s Performance Season again! Couple writing suggestions to ease next year’s conversations: 1….
It’s Performance Season again!
Couple writing suggestions to ease next year’s conversations:
Brag docs (/ht @b0rk for https://jvns.ca/blog/brag-documents/) are an excellent way to make next year’s review more accurate and complete. Better to prune 10 months from now than reconstruct what happened 10 months ago.
Work-product design docs linked from your primary doc benefit from the USENIX guidance for writing good Systems Papers: https://www.usenix.org/conferences/author-resources/how-and-how-not-write-good-systems-paper
Mastodon Source 🐘
@b0rk Design docs that call out paths not taken. Context referenced for a decision and then discarded is context most likely to be forgotten. Help the future reader understand the conditions, criteria, and your judgment at the time.
“What were the alternatives considered at various points, and why were the choices made the way they were? A good paper doesn’t just describe, it explains. Telling your readers what you did doesn’t give them any idea how carefully considered your choices were.”