In the middle of the desert you can say anything you want
ssh -v localhost is a quick way to get the versions of everything.
Here and later, ‘host’ is the thingy hidden behind NAT.
cloudflared on the server
How to see ping requests being recieved on the destination machine? - Super User:
Wireshark is too heavy duty for something so simple. Just use
tcpdump -nn icmp. Add andhost 1.2.3.4if you want to limit it to packets coming from 1.2.3.4.
I religiously do .realpath() pretty much every time I get a path from user input. Naively believing it also expands ~ etc.
Once I forgot and once I entered a non-expanded path myself: ~/this/
Then was tracking it as a bug, and found this bundle of joy:
/home/sh/me/dir~/me/dir/Checkpoints/checkpoint_288
It is in fact not illegal to create a directory called ~ in Unix.
And the things that used it as-is where there, and the things that were using it after a realpath were using another directory.
OK, I resolve()-d it - still the same.
TIL Path.resolve() takes care of symlinks and ..-like components, but not ~. So it should be Path.expanduser().resolve() from now on.
jq’s to_entries allows parsing key names as values/fiels:
``s__` jq ’to_entries' Input {“a”: 1, “b”: 2} Output [{“key”:“a”, “value”:1}, {“key”:“b”, “value”:2}]
Documented worse than I’d like to.
Filters allow to do things to the records (structs that make up a log message later), be it change them in place or don’t let them pass.
You can pass a function in place of a Filter, it should:
logging.LogRecordThe fields of a LogRecord are the same ones we name when doing formatting: name, lineno, msg and friends.
If your Filter tries to log something in a way that it’ll get filtered through it, you get recursion.
Sample of a filter that removes specific matches and gets added to a Handler:
def filter(record: logging.LogRecord) -> int:
"""Filters away log records containing annoying stuff."""
blacklist_condition = (
(
record.name == "lib_sdk.data"
and "not available on your" in record.msg
)
or (
record.name == "lib_sdk.data"
and record.levelno == logging.WARNING
and "which is legacy" in record.msg
)
or (
record.name == "lib_sdk.data"
and record.levelno == logging.WARNING
and "created but without information" in record.msg
)
)
if blacklist_condition:
return 0
else:
return 1
sh = logging.StreamHandler()
sh.addFilter(filter)
Much better than what I had before (220914-2249 Python logging change level through context manager and operator magic).
One can go crazy here with regexes etc. but I shan’t.
Goal: log everything to file, but show only part of the info on the screen. Previously: 220914-2249 Python logging change level through context manager and operator magic
My current understanding:
format = "[%(asctime)s %(name)s:%(lineno)s %(levelname)s]: %(message)s"
# Set it up, no handlers -> no default StreamHandler
# this loglevel is the one handlers will have access to!
logging.basicConfig(
level=logging.DEBUG,
handlers=[]
)
# Format, if we don't do this will be literally none
fmtr = logging.Formatter(fmt=format)
sh = logging.StreamHandler()
fh = logging.FileHandler("debug.log")
fh.setFormatter(fmtr)
sh.setFormatter(fmtr)
# Screen output set to whatever we want, fh to debug
sh.setLevel(loglevel)
fh.setLevel(logging.DEBUG)
# Add both handlers to root, both get propagated to logger etc.
logging.getLogger('').addHandler(sh)
logging.getLogger('').addHandler(fh)
Even though i did logger = logging.getLogger(__package__) at the very top of the file before the above bits, I can do logger.debug() etc. and it follows these settings. Nice.
My standard logging setup is logger=logging.getLogger(__package__) in my main runner file and .getLogger(__name__) for all other files.
I wanted to temporarily change the loglevel of a specific logger of a library. Logical thing is to use a context manager, and such things exist:
I liked the second one, but what I wanted is to change the loglevel of another logger.
Usage:
# inside somelib.data...
liblogger = logging.getLogger(__name__)
logger.info("Stuff")
liblogger.info("Stuff from the lib")
with LoggingContext(
"somelib.data",
level=logging.ERROR
):
# very deep inside somelib.data...
liblogger.warning("Useless warning")
liblogger.warning("Not useless warning")
logger.info("Stuff")
Idea:
logger.debug()s in my worldBut if I’m debugging I want these useless warnings!
After doing level=logging.ERROR if logger.level != logging.DEBUG else logging.getLogger('somelib_data').level oneliners I decided that I want the context manager to be flexible.
Ended up with this:
class LoggingContext:
"""Temporarily change the loglevel of a logger based on loglevels of
other loggers or arbitrary conditions."""
def __init__(
self,
logger_name: str,
level_true: int,
level_false: Optional[int] = None,
l1: Union[logging.Logger, int] = logger,
l2: Optional[int] = None,
comp_fn: Optional[Callable] = lambda x, y: True,
):
"""Temporarily change logging level of a logger, optionally dependent
on another logger's level.
:param logger_name: Change the level of a logger with this name
if None, the `level` new logger level will be used
:param callable_for_unchanged: if set, will be used to compare
main_logger_level to comparison logger level
and if True, will leave everything unchanged.
:param level_true: which loglevel to set in logger if condition is True
:param level_false: loglevel to set if condition is False
None means "don't change anything"
:param l1: main logger whose effective loglevel we'll use, or a loglevel
if None the global `logger` will be used
:param l2: loglevel to compare l1 with
if None will compare to the loglevel `level_true`
:param comp_fn: callable taking two params, loglevels/ints l1 and l2,
returning a boolean. Can be a lambda function or `operators` library
operators (eq,neq etc.)
If None will return True, ergo setting level_true always
"""
self.other_logger = logging.getLogger(logger_name)
# If it's a logger, get its effective level, if int - use that
main_level = (
l1.getEffectiveLevel() if isinstance(l1, logging.Logger) else l1
)
# Compare to l2 if it's there, otherwise to level_true
effective_comparison_level = l2 if l2 else level_true
# If callable is True, leave everything unchanged
comparison_result = comp_fn(main_level, effective_comparison_level)
# If we have no level_false, interpret it as "don't change anything"
if comparison_result:
self.level = level_true
else:
# 'None' here is a magic value "don't change anything"
self.level = level_false
logger.debug(
f"{logger_name=}, {l1=}, {l2=}, "
f"{level_true=}, {level_false=}, {comp_fn=}"
)
logger.debug(
f"{self.other_logger=}, {self.level=}, {main_level=}, "
f"{effective_comparison_level=}, {comparison_result=}"
)
if self.level is not None:
logger.debug(f"Changing {logger_name=} to loglevel {self.level}")
else:
logger.debug(f"Leaving {logger_name=} unchanged.")
def __enter__(self):
if self.level is None:
return None
self.old_level = self.other_logger.level
self.other_logger.setLevel(self.level)
def __exit__(self, et, ev, tb):
if self.level is None:
return None
else:
self.other_logger.setLevel(self.old_level)
This changes the idea completely and brings some VERY non-intuitive dynamics with default values, not sure yet if it’s worth doing it like that for the sake of brevity but we’ll see.
level_true, level_false are levels to use based on conditionl1, l2 are the two loglevels we comparecond_fn is a Callable/lambda/… that does the condition and returns a boolean.level_false means “no change to status quo”l1 takes the global logger, which is probably a child of the logger we care about and inherits its effective loglevell2 becomes level_true
l1”with LoggingContext('other', logging.ERROR):with LoggingContext('other', logging.INFO, comp_fn=operators.le):with LoggingContext('other', logging.ERROR,
l2=logging.DEBUG, comp_fn=operators.eq):
from operators import le as less_or_equal
with LoggingContext('other', level_true=logging.WARNING,
level_false=logging.ERROR,
l1=logger.level, # just as demo, it's implicit everywhere
l2=logging.INFO, comp_fn=less_or_equal):`
Initially it was lambdas, but I kept wishing for “can I just pass <= as a function?” and lo and behold - yes, through the operator library!
That was fun, and TIL about operators. In any case - another function for my small library of snippets.
Best of all, my favourite python blog has an article about the topic:The Unknown Features of Python’s Operator Module | Martin Heinz | Personal Website & Blog
Let’s see if I end up using this utility function more than once.
Another similar-ish snippet I wrote once and still love. You get pretty progress bars only if you have enough elements in your sequence for it to make sense:
def _tqdm(list_like: Sequence, iflarge: bool = False, lsize: int = 100, **kwargs):
"""Use tqdm if it's on, optionally based on length of list.
Args:
list_like: thing to iterate.
iflarge (bool): If on, will use tqdm only for large lists
lsize (int): anything more than this is 'large'
**kwargs: get passed to tqdm as-is
"""
if USE_TQDM:
if not iflarge:
return tqdm(list_like, **kwargs)
else:
# Count size only if it doesn't mean iterating an iterator
if isinstance(list_like, Sequence) and len(list_like) > lsize:
return tqdm(list_like, **kwargs)
return list_like
Then, if the global USE_TQDM is true:
for x in _tqdm(sth) is a vanilla tqdmfor x in _tqdm(sth, True) becomes a tqdm only if we’re iterating through something larger than 100 elements._tqdm(sth, True, 50, desc="DOCS") tqdm on 50+ elements with a label (how cool is that?)And on the same topic:
def log(msg) -> None:
"""Use loglevel.debug if tqdm is used, loglevel.info otherwise."""
if USE_TQDM:
logger.debug(msg)
else:
logger.info(msg)
logger.info() destroy tqdms, so - if we’re using TQDM, log it as logger.debug(). We’ll still see it on that loglevel if we want to (or maybe we’re logging it to a file, who knows).
In .ideavimrc I added these two:
nmap <leader><leader> :action CloseContent<cr>
nmap <C-S-T> :action ReopenClosedTab<cr>
First equal to my vim settings, second equal to the usual binding for it in “normal” browsers.
Python has a property function/decorator: Built-in Functions — Python 3.10.7 documentation.
Basically - you have a field and you want getter/setter functions on it.
Seen first in konfuzio_sdk, sample from there:
@property
def number_of_lines(self) -> int:
"""Calculate the number of lines in Page."""
return len(self.text.split('\n'))
Then you can run document.number_of_lines and it runs the function.