google reader refugee.
1750 stories
·
43 followers

my employees refuse to call their coworker by her real name

2 Comments and 5 Shares

A reader writes:

One of my long-term staff has a common, easy-to-pronounce Indian name, but since well before I was hired, she was given a nickname: a westernised version of her name. We were chatting about my (slightly unusual) name one day, and she expressed that she hates the nickname, wishes people would just use her real name, and that she’s never felt confident asking people to do so. I offered, as her manager, to handle this for her, and she agreed, stating that she’d be grateful.

Responses were mixed but generally negative, and many of the team are refusing to call her anything but the nickname. The general consensus is that it’s “prettier” or that her name “isn’t very feminine.” When asked directly, she finds it difficult to be rude, so will only say that she prefers her full name. It’s now at the point where I’m having to inform my new senior manager that the nickname isn’t appropriate, because staff members have informed her that the nickname is preferred.

I had a conversation today with one team member about this, and she informed me that unless she’s told by the person with the nickname that she “only wants to be called by her other name,” she will continue to use the nickname when speaking about her to coworkers or clients, or directly to her. I feel that this is vastly inappropriate, but without my staff member having the confidence to address this more strongly, there doesn’t seem to be much I can do. That said, it seems disrespectful at the very least.

Should I push further on what, to most of the team, is a minor issue, or let it go and hope that my team member can stand up for herself?

This reminds me of last week’s letter from the manager whose employee was harassing a coworker about her prosthetic limb.

That manager needed to use her authority to put a stop to something offensive, and so do you. You don’t need to talk anyone into behaving respectfully; you need to tell them that it’s not optional.

Your staff members’ behavior here is, frankly, disgusting. They want to westernize someone’s name because her actual name isn’t “pretty” or “feminine” enough for them? No. That’s not an option, they’re being offensive and racist, offensive, and you need to require them to behave respectfully and not like the giant assholes they’re currently being.

Talk with each of the offenders individually and say this: “We’ve talked about this before and I erred in not being clear enough — Parvati’s name isn’t Polly; it’s Parvati. She’s asked that we use her correct name, and that’s what you need to call her going forward. I need you to be vigilant about respecting that request and calling her Parvati from now on.”

If you get any of this crap about “not unless she tells me herself that she only wants to be called by her real name,” stamp that out immediately. Say this: “No. I’m telling you clearly right now that she has asked to be called Parvati, and that I expect you to do that — and I expect you to do that without giving her any trouble about it. Can you agree to do that?”

You’re asking that last part — “can you agree to do that?” — because you want the person to commit to it here and now … and if they’re still reluctant, you want to find that out before you end the conversation.

But you absolutely, 100% need to do this. Your employee has told you very clearly that she prefers her given name, and you cannot allow her colleagues to decide to westernize her name for their own comfort. Get it stopped today. Seriously, this is horrible.

my employees refuse to call their coworker by her real name was originally published by Alison Green on Ask a Manager.

Read the whole story
pfctdayelise
1 day ago
reply
Melbourne, Australia
Share this story
Delete
2 public comments
Courtney
1 day ago
reply
I swear to fucking god, for all the abuses of power you hear about, then there's stuff like this where it's like "YOU ARE THE BOSS. BOSS PEOPLE AROUND. LITERALLY YOUR JOB."
Portland, OR
duerig
18 hours ago
Exactly. "The best lack all conviction while the worst are full of passionate intensity.". *sigh*
ryanbrazell
1 day ago
reply
Today in "people are the WORST."
Richmond, VA

Testing, for people who hate testing

2 Shares

I love having tests.

I hate writing them.

It’s tedious. It’s boring. It’s hard, sometimes harder than writing the code. Worst of all, it doesn’t feel like it accomplishes anything.

So I usually don’t do it. I know, I know. I should do it. I should also get more exercise and eat more vegetables.

The funny thing is, the only time I see anyone really praise the benefits of testing is when someone who’s really into testing extols the virtues of test-driven development. To me, that’s like trying to get me to eat my veggies by telling me how great veganism is. If I don’t want to do it at all, trying to sell me on an entire lifestyle is not going to work. I need something a little more practical, like “make smoothies” or “technically, chips are a vegetable”.

Here’s the best way I’ve found to make test smoothies. I’ll even deliberately avoid any testing jargon, since no one can agree on what any of it means anyway.

Hating testing less

Love your test harness

Your test harness is the framework that finds your tests, runs your tests, and (in theory) helps you write tests.

If you hate your test harness, you will never enjoy writing tests. It’ll always be a slog, and you’ll avoid it whenever you can. Shop around and see if you can find something more palatable.

For example, Python’s standard solution is the stdlib unittest module. It’s a Java-inspired monstrosity that has you write nonsense like this:

1
2
3
4
5
6
7
import unittest

from mymodule import whatever

class TestWhatever(unittest.TestCase):
    def test_whatever(self):
        self.assertIn(whatever(), {1, 2, 3})

It drives me up the wall that half of this trivial test is weird boilerplate. The class itself is meaningless, and the thing I really want to test is obscured behind one of several dozen assert* methods.

These are minor gripes, but minor gripes make a big difference when they apply to every single test — and when I have to force myself to write tests in the first place. (Maybe they don’t bother you, in which case, keep using unittest!) So I use py.test instead:

1
2
3
4
from mymodule import whatever

def test_whatever(self):
    assert whatever() in {1, 2, 3}

If the test fails, you still get useful output, including diffs of strings or sequences:

1
2
3
4
    def test_whatever():
>       assert whatever() in {1, 2, 3}
E       assert 4 in set([1, 2, 3])
E        +  where 4 = whatever()

You really, really don’t want to know how this works. It does work, and that’s all I care about.

py.test also has some bells and whistles like the ability to show locals when a test fails, hooks for writing your own custom asserts, and bunches of other hooks and plugins. But the most important thing to me is that it minimizes the friction involve in writing tests to as little as possible. I can pretty much copy/paste whatever I did in the REPL and sprinkle asserts around.

If writing tests is hard, that might be a bug

I’ve seen tests that do some impressive acrobatics just to construct core objects, and likewise heard people grumble that they don’t want to write tests because creating core objects is so hard.

The thing is, tests are just code. If you have a hard time constructing your own objects with some particular state, it might be a sign that your API is hard to use!

Well, we never added a way to do this because there’s no possible reason anyone would ever want it.” But you want it, right now. You’re consuming your own API, complaining that it can’t do X, and then not adding the ability to do X because no one would ever need X.

One of the most underappreciated parts of writing tests is that they force you to write actual code that uses your interfaces. If doing basic setup is a slog, fix those interfaces.

Aggressively make your test suite fast and reliable

I’ve worked with test suites that took hours to run, if they ran at all.

The tradeoff is obvious: these test suites were fairly thorough, and speed was the cost of that thoroughness. For critical apps, that might be well worth it. For very large apps, that might be unavoidable.

For codebases that are starting out with no tests at all, it’s a huge source of testing pain. Your test suite should be as fast as possible, or you won’t run it, and then you’ll (rightfully!) convince yourself that there’s no point in writing even more tests that you won’t run.

If your code is just slow, consider this an excellent reason to make it faster. If you have a lot of tests, see if you can consolidate some.

Or if you have a handful of especially slow tests, I have a radical suggestion: maybe just delete them. If they’re not absolutely critical, and they’re keeping you from running your test suite constantly, they may not be worth the cost. Deleting a test drops your coverage by a fraction of a percent; never running your tests drops your coverage to zero.

Flaky tests are even worse. Your tests should always, always, pass completely. If you have a test that fails 10% of the time and you just can’t figure out why, disable or delete it. It’s not telling you anything useful, and in the meantime it’s training you to ignore when your tests fail. If a failing test isn’t an immediate red alert, there’s no point in having tests at all.

Run it automatically

Have you seen those GitHub projects where pull requests automatically get a thing saying whether the test suite passed or failed? Neat, right? It’s done through Travis, and it’s surprisingly painless to set up. Once it is set up, someone else’s computer will run your tests all the damn time and bug you when they fail. It’s really annoying, and really great.

(There’s also Coveralls, which measures your test coverage. Neat, but if you’re struggling to write tests at all, a looming reminder of your shame may not be the most helpful thing.)

I recently ran into an interesting problem in the form of Pelican, the Python library that generates this blog. It has tests for the fr_FR locale, and the test suite skips them if you don’t have that locale set up… but the README tells you that before you submit a pull request, you should generate the locale so you can run the tests. Naturally, I missed this, didn’t have fr_FR, thought I passed all the tests, and submitted a pull request that instantly failed on Travis.

Skipping tests because optional dependencies are missing is a tricky affair. When you write them, you think “no point claiming the test failed when it doesn’t indicate problems with the actual codebase” — when I run them, I think “oh, these tests were skipped, so they aren’t really important”.

What to test

Test what you manually test

When you’re working on a big feature or bugfix, you develop a little ritual for checking whether it’s done. You crack open the REPL and repeat the same few lines, or you run a script you hacked together, or your launch your app and repeat the same few actions. It gets incredibly tedious.

You may have similar rituals just before a big release: run the thing, poke around a bit, try common stuff, be confident that at least the basics work.

These are the best things to test, because you’re already testing them! You can save yourself a lot of anguish if you convert these rituals into code. As an added benefit, other people can then repeat your rituals without having to understand yours or invent their own, and your test suite will serve as a rough description of what you find most important.

Sometimes, this is hard. Give it a try anyway, even if (especially if) you don’t have a test suite at all.

Sometimes, this is really hard. Write tests for the parts you can, at least. You can always sit down and work the rest out later.

Test what's likely to break

Some things are easy to test. If you have a function that checks whether a number is even, oh boy! You can write like fifty tests for that, no problem. Now you have fifty more tests! Good job!

That’s great, and feel free to write them all, but… how likely is it that anyone will ever change that function? It does one trivial thing, it can be verified correct at a glance, it doesn’t depend on anything else, and it almost certainly can’t be improved upon.

The primary benefit of testing is a defense against change. When code changes, tests help convince you that it still works correctly. Testing code that has no reason to change doesn’t add much more value to your test suite.

This isn’t to say that you shouldn’t test trivial functions — especially since we can be really bad at guessing what’ll change in the future — but when you have limited willpower, they’re not the most efficient places to spend it.

Knowing what to test is the same kind of artform as knowing what to comment, and I think many of the same approaches apply. Test obscure things, surprising special cases. Test things that were tricky to get right, things that feel delicate. Test things that are difficult to verify just by reading the code. If you feel the need to explain yourself, it’s probably worth testing.

Test the smallest things you can possibly test

It’s nice to have some tests that show your code works from a thousand miles up. Unfortunately, these also tend to be the slowest (because they do a lot), most brittle (because any small change might break many such tests at once), least helpful (because a problem might come from anywhere), and least efficient (because two such tests will run through much of the same code).

People who are into testing like to go on about unit tests versus functional tests, or maybe those are integration tests, or are they acceptance tests, or end-to-end tests, or… christ.

Forget the categories. You already know the shape of your own codebase: it’s a hierarchy of lumps that each feel like they relate to a particular concept, even if the code organization doesn’t reflect that. You’re writing a disassembler, and there’s some code in various places that deals with jumps and labels? That’s a lump, even if the code isn’t contiguous on disk. You, the human, know where it is.

So write your tests around those lumps, and make them as small as possible. Maybe you still have to run your entire disassembler to actually run certain tests, but you can still minimize the extra work: disable optional features and make the test as simple as possible. If you ever make changes to jumps or labels, you’ll know exactly which tests to look for; if those tests ever break, you’ll have a good idea of why.

Don’t get me wrong; I know it’s reassuring to have a mountain of tests that run through your entire app from start to finish, just as a user would. But in my experience, those tests break all the time without actually telling you anything you didn’t already know, and having more than a handful of them can bog down the entire test suite. Hesitate before each one you write.

How to test

Test output, avoid side effects

Testing code should be easy. You make some input; you feed it to a function; you check that the output is correct. The precise nature of “input” and “output” can easily spiral out of control, but at least the process is simple enough.

Testing code that has side effects is a huge, huge pain in the ass. (And since tests are just code, that means using code that has side effects is also a pain in the ass.)

Side effect” here means exactly what it sounds like: you feed input into a function, you get some output, and in the meantime something elsewhere has changed. Or in a similar vein, the behavior of the function depends on something other than what’s passed into the function. The most common case is global record-keeping, like app-wide configuration that sits at the module level somewhere.

It sucks, it’s confusing, avoid avoid avoid.

So… don’t use globals, I guess?

I’ve heard a lot of programmers protest that there’s nothing very difficult to understand about one global, and I’m going to commit heresy here by admitting: that’s probably true! The extra cognitive burden of using and testing code that relies on a single global is not particularly high.

But one global begets another, and another. Or perhaps your “one” global mutates into a massive sprawling object with tendrils in everything. Soon, you realize you’ve written the Doom renderer and you have goofy obscure bugs because it’s so hard to keep track of what’s going on at any given time.

Similar to the much-maligned C goto, globals aren’t an infectious and incurable toxin that will instantly and irreparably putrefy your codebase. They just have a cost, and you’ll only need to pay it sometime down the road, and it’s usually not worth the five minutes of effort saved. If you must introduce a global, always take a moment to feel really bad about what you’re doing.

Test negative cases and edge cases

I used to work for a company. As part of the hiring process, prospective hires would be asked to implement a particular board game, complete with tests. Their solution would be dumped in my lap, for some reason, and I would unleash the harsh light of my judgment upon it.

I’m being intentionally vague because I don’t want to help anyone cheat, any more than I already am by telling you how to write tests. So let’s say the game is the most trivial of board games: tic tac toe.

A significant proportion of solutions I graded had test suites like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
board = """
    X--
    -X-
    --X
"""
assert check_winner(board) == "X"

board = """
    OOO
    ---
    ---
"""
assert check_winner(board) == "O"

And that was it. Two or three tests for particular winning states. (Often not even any tests for whether placing a piece actually worked, but let’s leave that aside.)

I would always mark down for this. The above tests check that your code is right, but they don’t check that your code isn’t wrong. What if there’s no winner, but your code thinks there is?

That’s much harder to reason about, I grant you! A tic tac toe board only has a relatively small handful of possible winning states, but a much larger number of possible non-winning states. But I think what really throws us is that winning is defined in the positive — “there must be three in a row” — whereas non-winning is only defined as, well, not winning. We don’t tend to think of the lack of something as being concrete.

When I took this test, I paid attention to the bugs I ran into while I was writing my code, and I thought about what could go wrong with my algorithm, and I made a few tests based on those educated guesses. So perhaps I’d check that these boards have no winner:

1
2
3
OO-     -O-     -X-
O-X     -O-     --X
-XX     -X-     X--

The left board has three of each symbol, but not in a row. The middle board has three in a row, but not all the same symbol. The right board has three in a row, but only if the board is allowed to wrap.

Are these cases likely to be false positives? I have no idea. All I did was consider for a moment what could go wrong, then make up some boards that would have that kind of error. (One or two of the solutions I graded even had the kinds of false positives that I’d written tests for!)

The same kind of thinking — what could I have missed? — leads me swiftly to another glaring omission from this test suite: what if there’s a tie? And, indeed, quite a few of the submissions I graded didn’t handle a tie at all. (Ties were less likely in the actual game than they are in tic tac toe, but still possible.) The game would ask you to make a move, you wouldn’t be able to, and the game would linger there forever.

Don’t just write tests to give yourself the smug satisfaction that you did something right. Write tests to catch the ways you might conceivably have done something wrong. Imagine the code you’re testing as an adversary; how might you catch it making a mistake?

If that doesn’t sound convincing, let me put this another way. Consider this hypothetical test suite for a primality test.

1
2
3
4
5
6
7
def test_is_prime():
    assert is_prime(2)
    assert is_prime(3)
    assert is_prime(5)
    assert is_prime(11)
    assert is_prime(17)
    assert is_prime(97)

Quick: write some code that passes these tests. Call it test-driven development practice.

Here’s what I came up with:

1
2
def is_prime(n):
    return True

Whoops!

A related benefit of negative tests is that they make sure your tests actually work. I’ve seen one or two tests that couldn’t reasonably verify that the output of some program was actually correct, so instead, they ran the program and checked that there were no errors. Later, something went wrong in the test suite, and the program silently didn’t run at all — which, naturally, produced no exceptions. A single test that fed in bad input and checked for an error would’ve caught this problem right away.

Refactor

Tests are code. If you’re repeating yourself a lot or there’s a lot of friction for some common task, refactor. Write some helpers. See if your test harness can help you out.

Tests are code. Don’t write a bunch of magical, convoluted, brittle garbage to power your tests. If you can’t convince yourself that your tests work, how can your tests convince you that the rest of your code works? You should be more confident in your tests than in the rest of your code, yet you’ll probably spend far less time maintaining it. So err on the side of explicit and boring, even if you have to stick to repeating yourself.

Troublesome cases

External state

Testing against something outside your program sucks. With filesystems, you can make a temporary directory. With time, you can (maybe) fake it. In general, your life will be easier if you consolidate all your external state access into as few places as possible — easy to understand, easy to test, easy to swap out for some alternative implementation.

With databases, you’re just fucked. Database access pervades most code that needs to touch a database at all.

The common wisdom in the Python web development community is that you should just run your test suite against a little SQLite database. That’s a great idea, except that you’re suddenly restricted to the subset of SQL that works identically in SQLite and in your target database. The next best thing is to run against an actual instance of your target database and call it a day.

And you should probably stop there; nothing I can come up with is any better. Even for very large apps with very complex databases, that seems to be the best you can do. You might end up spending twenty minutes per test run starting up a full replicated setup and memcached and whatever else, but I don’t have any better ideas.

The problem is that database access still goes through SQL, and SQL is an entire other programming language you’re sending out over the wire. You can’t easily swap in an in-process SQL implementation — that’s called SQLite. You can hide all database access in functions with extremely long names and convoluted return values that are only called in one place, then swap out a dummy implementation for testing, but that’s really no fun at all. Plus, it doesn’t check that your SQL is actually correct.

If you’re using an ORM, you have slightly more of a chance, but I’ve never seen an ORM that can natively execute queries against in-memory data structures. (I would love to, and it seems within the realm of possibility, but it would be a huge amount of work and still not cover all the little places you’re using functions and syntax specific to your database.)

I don’t know. I got nothin’.

Procedural generation and other randomness

Imagine you wrote NetHack, which generates some 2D cavern structures. How can you possibly test that the generated caverns are correct, when they’re completely random?

I haven’t gotten too deep into this, but I think there’s some fertile ground here. You don’t know exactly what the output should be, but you certainly have some constraints in mind. For example, a cavern map should be at least 10% cave walls and at least 30% open space, right? Otherwise it’s not a cavern. You can write a test that verifies that, then just run it some number of times.

You can’t be absolutely sure there are no edge cases (unless you are extremely clever in how you write the terrain generation in the first place), but each run of the test suite will leave you a little more confident. There’s a real risk of flaking here, so you’ll have to be extra vigilant about diagnosing and fixing any problems.

You can also write some more specific tests if you give your thing-generator as many explicit parameters as possible, rather than having it make all its decisions internally. Maybe your cavern algorithm takes a parameter for how much open space there is, from 0.3 to 0.9. If you dial it down to the minimum, will there still be an open path from the entrance to the exit? You can test for that, too.

Web output

This is kind of an interesting problem. HTML is more readily inspected than an image; you can parse it, drill down with XPath or CSS selectors or what have you, and check that the right text is in the right places.

But! You may also want to know that it looks right, and that’s much more difficult. The obvious thing is to automate a browser, take a screenshot, and compare it to a known good rendering — all of which will come crumbling down the moment someone makes a border one pixel wider. I don’t know if we can do any better, unless we can somehow explain to a computer what “looks right” means.

Something I’d like to see is an automated sanity check for HTML + CSS. Lay out the page without rendering it and check for any obvious screwups, like overlapping text or unwanted overflow. I don’t know how much practical use this would be (or whether it already exists), but it seems like a nice easy way to check that you didn’t do something catastrophic. You wouldn’t even necessarily need it in your test suite — just plug it into a crawler and throw it at your site.

GUIs and games

Oh my god I have no idea. Keep your UI separated from your internals, test the internals, and hope for the best.

But most of all

Just test something. Going from zero tests to one test is an infinite improvement.

Once you have a tiny stub of a test suite, you have something to build on, and the next test will be a little easier to write. You might even find yourself in the middle of adding a feature and suddenly thinking, hey! this is a great opportunity to write a quick test or two.

Read the whole story
acdha
22 hours ago
reply
Washington, DC
pfctdayelise
1 day ago
reply
Melbourne, Australia
Share this story
Delete

Python Packaging Is Good Now

2 Shares

Okay folks. Time’s up. It’s too late to say that Python’s packaging ecosystem terrible any more. I’m calling it.

Python packaging is not bad any more. If you’re a developer, and you’re trying to create or consume Python libraries, it can be a tractable, even pleasant experience.

I need to say this, because for a long time, Python’s packaging toolchain was … problematic. It isn’t any more, but a lot of people still seem to think that it is, so it’s time to set the record straight.

If you’re not familiar with the history it went something like this:

The Dawn

Python first shipped in an era when adding a dependency meant a veritable Odyssey into cyberspace. First, you’d wait until nobody in your whole family was using the phone line. Then you’d dial your ISP. Once you’d finished fighting your SLIP or PPP client, you’d ask a netnews group if anyone knew of a good gopher site to find a library that could solve your problem. Once you were done with that task, you’d sign off the Internet for the night, and wait about 48 hours too see if anyone responded. If you were lucky enough to get a reply, you’d set up a download at the end of your night’s web-surfing.

pip search it wasn’t.

For the time, Python’s approach to dependency-handling was incredibly forward-looking. The import statement, and the pluggable module import system, made it easy to get dependencies from wherever made sense.

In Python 2.01, Distutils was introduced. This let Python developers describe their collections of modules abstractly, and added tool support to producing redistributable collections of modules and packages. Again, this was tremendously forward-looking, if somewhat primitive; there was very little to compare it to at the time.

Fast forwarding to 2004; setuptools was created to address some of the increasingly-common tasks that open source software maintainers were facing with distributing their modules over the internet. In 2005, it added easy_install, in order to provide a tool to automate resolving dependencies and downloading them into the right locations.

The Dark Age

Unfortunately, in addition to providing basic utilities for expressing dependencies, setuptools also dragged in a tremendous amount of complexity. Its author felt that import should do something slightly different than what it does, so installing setuptools changed it. The main difference between normal import and setuptools import was that it facilitated having multiple different versions of the same library in the same program at the same time. It turns out that that’s a dumb idea, but in fairness, it wasn’t entirely clear at the time, and it is certainly useful (and necessary!) to be able to have multiple versions of a library installed onto a computer at the same time.

In addition to these idiosyncratic departures from standard Python semantics, setuptools suffered from being unmaintained. It became a critical part of the Python ecosystem at the same time as the author was moving on to other projects entirely outside of programming. No-one could agree on who the new maintainers should be for a long period of time. The project was forked, and many operating systems’ packaging toolchains calcified around a buggy, ancient version.

From 2008 to 2012 or so, Python packaging was a total mess. It was painful to use. It was not clear which libraries or tools to use, which ones were worth investing in or learning. Doing things the simple way was too tedious, and doing things the automated way involved lots of poorly-documented workarounds and inscrutable failure modes.

This is to say nothing of the fact that there were critical security flaws in various parts of this toolchain. There was no practical way to package and upload Python packages in such a way that users didn’t need a full compiler toolchain for their platform.

To make matters worse for the popular perception of Python’s packaging prowess2, at this same time, newer languages and environments were getting a lot of buzz, ones that had packaging built in at the very beginning and had a much better binary distribution story. These environments learned lessons from the screw-ups of Python and Perl, and really got a lot of things right from the start.

Finally, the Python Package Index, the site which hosts all the open source packages uploaded by the Python community, was basically a proof-of-concept that went live way too early, had almost no operational resources, and was offline all the dang time.

Things were looking pretty bad for Python.


Intermission

Here is where we get to the point of this post - this is where popular opinion about Python packaging is stuck. Outdated information from this period abounds. Blog posts complaining about problems score high in web searches. Those who used Python during this time, but have now moved on to some other language, frequently scoff and dismiss Python as impossible to package, its packaging ecosystem as broken, PyPI as down all the time, and so on. Worst of all, bad advice for workarounds which are no longer necessary are still easy to find, which causes users to pre-emptively break their environments where they really don’t need to.


From The Ashes

In the midst of all this brokenness, there were some who were heroically, quietly, slowly fixing the mess, one gnarly bug-report at a time. pip was started, and its various maintainers fixed much of easy_install’s overcomplexity and many of its flaws. Donald Stufft stepped in both on Pip and PyPI and improved the availability of the systems it depended upon, as well as some pretty serious vulnerabilities in the tool itself. Daniel Holth wrote a PEP for the wheel format, which allows for binary redistribution of libraries. In other words, it lets authors of packages which need a C compiler to build give their users a way to not have one.

In 2013, setuptools and distribute un-forked, providing a path forward for operating system vendors to start updating their installations and allowing users to use something modern.

Python Core started distributing the ensurepip module along with both Python 2.7 and 3.3, allowing any user with a recent Python installed to quickly bootstrap into a sensible Python development environment with a one-liner.

A New Renaissance

I won’t give you a full run-down of the state of the packaging art. There’s already a website for that. I will, however, give you a précis of how much easier it is to get started nowadays. Today, if you want to get a sensible, up-to-date python development environment, without administrative privileges, all you have to do is:

1
2
3
$ python -m ensurepip --user
$ python -m pip install --user --upgrade pip
$ python -m pip install --user --upgrade virtualenv

Then, for each project you want to do, make a new virtualenv:

1
2
3
$ python -m virtualenv lets-go
$ . ./lets-go/bin/activate
(lets-go) $ _

From here on out, now the world is your oyster; you can pip install to your heart’s content, and you probably won’t even need to compile any C for most packages. These instructions don’t depend on Python version, either: as long as it’s up-to-date, the same steps work on Python 2, Python 3, PyPy and even Jython. In fact, often the ensurepip step isn’t even necessary since pip comes preinstalled. Running it if it’s unnecessary is harmless, even!

Other, more advanced packaging operations are much simpler than they used to be, too.

  • Need a C compiler? OS vendors have been working with the open source community to make this easier across the board:
    1
    2
    3
    4
    5
    $ apt install build-essential python-dev # ubuntu
    $ xcode-select --install # macOS
    $ dnf install @development-tools python-devel # fedora
    C:\> REM windows
    C:\> start https://www.microsoft.com/en-us/download/details.aspx?id=44266
    

Okay that last one’s not as obvious as it ought to be but they did at least make it freely available!

  • Want to upload some stuff to PyPI? This should do it for almost any project:

    1
    2
    3
    $ pip install twine
    $ python setup.py sdist bdist_wheel
    $ twine upload dist/*
    
  • Want to build wheels for the wild and wooly world of Linux? There’s an app4 for that.

Importantly, PyPI will almost certainly be online. Not only that, but a new, revamped site will be “launching” any day now3.

Again, this isn’t a comprehensive resource; I just want to give you an idea of what’s possible. But, as a deeply experienced Python expert I used to swear at these tools six times a day for years; the most serious Python packaging issue I’ve had this year to date was fixed by cleaning up my git repo to delete a cache file.

Work Still To Do

While the current situation is good, it’s still not great.

Here are just a few of my desiderata:

  • We still need better and more universally agreed-upon tooling for end-user deployments.
  • Pip should have a GUI frontend so that users can write Python stuff without learning as much command-line arcana.
  • There should be tools that help you write and update a setup.py. Or a setup.python.json or something, so you don’t actually need to write code just to ship some metadata.
  • The error messages that you get when you try to build something that needs a C compiler and it doesn’t work should be clearer and more actionable for users who don’t already know what they mean.
  • PyPI should automatically build wheels for all platforms by default when you upload sdists; this is a huge project, of course, but it would be super awesome default behavior.

I could go on. There are lots of ways that Python packaging could be better.

The Bottom Line

The real takeaway here though, is that although it’s still not perfect, other languages are no longer doing appreciably better. Go is still working through a number of different options regarding dependency management and vendoring, and, like Python extensions that require C dependencies, CGo is sometimes necessary and always a problem. Node has had its own well-publicized problems with their dependency management culture and package manager. Hackage is cool and all but everything takes a literal geological epoch to compile.

As always, I’m sure none of this applies to Rust and Cargo is basically perfect, but that doesn’t matter, because nobody reading this is actually using Rust.

My point is not that packaging in any of these languages is particularly bad. They’re all actually doing pretty well, especially compared to the state of the general programming ecosystem a few years ago; many of them are making regular progress towards user-facing improvements.

My point is that any commentary suggesting they’re meaningfully better than Python at this point is probably just out of date. Working with Python packaging is more or less fine right now. It could be better, but lots of people are working on improving it, and the structural problems that prevented those improvements from being adopted by the community in a timely manner have almost all been addressed.

Go! Make some virtualenvs! Hack some setup.pys! If it’s been a while and your last experience was really miserable, I promise, it’s better now.


Am I wrong? Did I screw up a detail of your favorite language? Did I forget to mention the one language environment that has a completely perfect, flawless packaging story? Do you feel the need to just yell at a stranger on the Internet about picayune details? Feel free to get in touch!


  1. released in October, 2000 

  2. say that five times fast. 

  3. although I’m not sure what it means to “launch” when the site is online, and running against the production data-store, and you can use it for pretty much everything... 

  4. “app” meaning of course “docker container” 

Read the whole story
pfctdayelise
1 day ago
reply
Melbourne, Australia
acdha
1 day ago
reply
Washington, DC
Share this story
Delete

whoever told you to be creative in your cover letter has led you horribly astray

2 Comments

A reader sent me this real-life cover letter that she received from a job applicant:

Hi. This very professionally-beginning cover letter should immediately alert you to my easy-going and comprehensive approach to task completion while making it strikingly obvious that I am both interested in and capable of performing HR duties with sophisticated exactitude. The quickening of your heart is likely subsiding at this point as your mind digests the familiar words in this second, poetic, figuration of how it feels to work with me: A profound calm washes over you as you realize we’re on the same team, striving for common goals. Clearly my efforts are oriented always toward mutually-beneficial understanding(s). As we part, smiling (of course), you check your pockets, laughing somewhat cynically at yourself for even thinking that I may’ve stolen your cell phone or wallet while we spoke, but then…your watch! Oh yes, you left it on the nightstand today, and come to think of it, you wrist feels so free and graceful, perhaps even sensual against the cuff of your shirt’s random meanderings. This, my friend, is how your life will feel every day we work together, except better.

Thank you.

I don’t know where to start, although not raising the possibility that you might pick your interviewer’s pocket is one place. Not invoking the sensual feeling of her wrist is another.

I know cover letters feel like an intimidating and even mysterious thing to many people. But really, they are just intended to explain why you’re interested in the job and why you’d excel at it. They really, really don’t need to do … whatever this letter is doing.

I would like to think this is a joke, but it’s very much a real thing that some job applicants have internalized the idea that they must do Something Different from everyone else in order to stand out, and often that manifests in creepy or otherwise bizarre ways.

whoever told you to be creative in your cover letter has led you horribly astray was originally published by Alison Green on Ask a Manager.

Read the whole story
pfctdayelise
7 days ago
reply
omg the wrist thing
Melbourne, Australia
Share this story
Delete

dealing with catcalls while I’m walking with coworkers

1 Comment and 3 Shares

A reader writes:

I’m a 20-year-old summer intern at a company that I absolutely love, and one of the reasons I like it so much is the casual environment and culture. There’s an office candy stash, complimentary yoga classes, and no dress code (my offer letter literally says that I could come to work in a sparkly cowboy suit if I really wanted to).

Usually, I wear sundresses (like many women in the office). Now, I don’t think this should be relevant, but I know some people might wonder — my dresses are nothing that would cause me to get a detention at the Catholic school I went to. That is, my skirt is always longer than my fingertips resting at my side, and the straps are always more than two inches thick. Despite the fact that this is my first office job, I can say with confidence that for the work environment I’m in, my attire is appropriate and professional. I look like everyone else, but a little younger.

Being able to dress for warm weather is especially useful in the summertime, because our office is actually split across two buildings that are a couple blocks apart, and the nature of my work means that at least once a day, I walk with coworkers from one building to the next. These blocks that we’re walking down are somewhat busy city blocks right next to a park, and I’ve had the same embarrassing thing happen to me a few times while walking: I’ve been catcalled by some passing jerk.

They’re nothing out of the ordinary for catcalls: “Hey there, pretty baby” and “Ooh, sexy!” and, most embarrassingly, “Hey red dress, what does that mouth do?” Euugh. When I’m alone or with a friend when I get catcalled, I just ignore it (or, admittedly, show the guy in question a particular finger). But I have no idea what to do when I’m with coworkers! It makes everything feel so awkward all of a sudden — conversations stop, eye contact is avoided — and I want to be able to defuse the tension. Staying silent feels like I’m not condemning it enough. Responding to the catcall feels too aggressive. Saying I’m embarrassed feels like an apology, when I haven’t done anything wrong. I once said something like “Ugh! Cat-callers are the worst!” but that didn’t really make the situation any less uncomfortable.

What can/should I do? Pretend it isn’t happening? Stop walking with my coworkers? Bite the bullet and cover up more at work (even though, I might add, this has even happened on days when I wear jeans and a t-shirt)? I’m an intern still getting used to the professional world, so I appreciate any advice you can give me.

Ugh, I’m sorry you’re dealing with this, and that it’s making you feel like the burden is on you to smooth things over for the people you’re with.

Ideally, your coworkers would be jumping in to support you, or at least ensuring there’s no weird silence. “Gross,” “what an asshole,” “that’s not okay,” or “ick, sorry that happened” would all be acceptable things for them to say. I’m not going to blame them too much, because it’s pretty normal for people not to have the perfect on-the-spot reaction in the face of this kind of thing, but I’m pointing it out so that you know that the responsibility doesn’t lie with you to make everyone else feel comfortable. If anyone is taking that on, it should be them for you.

As for what you should do in the moment: whatever’s going to make you the most comfortable. “Gross,” “what an asshole,” and “ick” are all perfectly appropriate things for you to say too. Or you can just continue on with whatever conversation you were having. Or you can say nothing, if that’s what you end up with! Saying nothing is not in any way an insufficient response. You don’t have any obligation here at all, and saying nothing is perfectly fine. Of everyone involved in this situation, you are the person with the least obligation to defuse the tension caused by someone harassing you.

For what it’s worth, your coworkers almost certainly know that this a crappy thing that happens to women, they’re not thinking you’re in any way responsible for it, and many of them probably deal with it themselves. They definitely don’t expect you to smooth it over; they’re almost certainly feeling awful that you’re having to deal with it and probably are questioning themselves later about whether they could have handled it better in the moment themselves.

And no, don’t conclude you need to dress differently or stop walking with your coworkers. You’re dressing appropriately for your office, and you don’t need to change your clothing, your perfectly normal habits, or your work relationships because of assholes.

dealing with catcalls while I’m walking with coworkers was originally published by Alison Green on Ask a Manager.

Read the whole story
pfctdayelise
15 days ago
reply
Fuck this shit
Melbourne, Australia
Share this story
Delete

What problem does it solve?

1 Share

One of the more puzzling aspects of Python for newcomers to the language is the stark usability differences between the standard library's urllib module and the popular (and well-recommended) third party module, requests, when it comes to writing HTTP(S) protocol clients. When your problem is "talk to a HTTP server", the difference in usability isn't immediately obvious, but it becomes clear as soon as additional requirements like SSL/TLS, authentication, redirect handling, session management, and JSON request and response bodies enter the picture.

It's tempting, and entirely understandable, to want to chalk this difference in ease of use up to requests being "Pythonic" (in 2016 terms), while urllib has now become un-Pythonic (despite being included in the standard library).

While there are certainly a few elements of that (e.g. the property builtin was only added in Python 2.2, while urllib2 was included in the original Python 2.0 release and hence couldn't take that into account in its API design), the vast majority of the usability difference relates to an entirely different question we often forget to ask about the software we use: What problem does it solve?

That is, many otherwise surprising discrepancies between urllib/urllib2 and requests are best explained by the fact that they solve different problems, and the problems most HTTP client developers have today are closer to those Kenneth Reitz designed requests to solve in 2010/2011, than they are to the problems that Jeremy Hylton was aiming to solve more than a decade earlier.

It's all in the name

To quote the current Python 3 urllib package documentation: "urllib is a package that collects several modules for working with URLs".

And the docstring from Jeremy's original commit message adding urllib2 to CPython: "An extensible library for opening URLs using a variety [of] protocols".

Wait, what? We're just trying to write a HTTP client, so why is the documentation talking about working with URLs in general?

While it may seem strange to developers accustomed to the modern HTTPS+JSON powered interactive web, it wasn't always clear that that was how things were going to turn out.

At the turn of the century, the expectation was instead that we'd retain a rich variety of data transfer protocols with different characteristics optimised for different purposes, and that the most useful client to have in the standard library would be one that could be used to talk to multiple different kinds of servers (like HTTP, FTP, NFS, etc), without client developers needing to worry too much about the specific protocol used (as indicated by the URL schema).

In practice, things didn't work out that way (mostly due to restrictive institutional firewalls meaning HTTP servers were the only remote services that could be accessed reliably), so folks in 2016 are now regularly comparing the usability of a dedicated HTTP(S)-only client library with a general purpose URL handling library that needs to be configured to specifically be using HTTP(S) before you gain access to most HTTP(S) features.

When it was written, urllib2 was a square peg that was designed to fit into the square hole of "generic URL processing". By contrast, most modern client developers are looking for a round peg to fit into the round hole that is HTTPS+JSON processing - urllib/urllib2 will fit if you shave the corners off first, but requests comes pre-rounded.

So why not add requests to the standard library?

Answering the not-so-obvious question of "What problem does it solve?" then leads to a more obvious follow-up question: if the problems that urllib/ urllib2 were designed to solve are no longer common, while the problems that requests solves are common, why not add requests to the standard library?

If I recall correctly, Guido gave in-principle approval to this idea at a language summit back in 2013 or so (after the requests 1.0 release), and it's a fairly common assumption amongst the core development team that either requests itself (perhaps as a bundled snapshot of an independently upgradable component) or a compatible subset of the API with a different implementation will eventually end up in the standard library.

However, even putting aside the misgivings of the requests developers about the idea, there are still some non-trivial system integration problems to solve in getting requests to a point where it would be acceptable as a standard library component.

In particular, one of the things that requests does to more reliably handle SSL/TLS certificates in a cross-platform way is to bundle the Mozilla Certificate Bundle included in the certifi project. This is a sensible thing to do by default (due to the difficulties of obtaining reliable access to system security certificates in a cross-platform way), but it conflicts with the security policy of the standard library, which specifically aims to delegate certificate management to the underlying operating system. That policy aims to address two needs: allowing Python applications access to custom institutional certificates added to the system certificate store (most notably, private CA certificates for large organisations), and avoiding adding an additional certificate store to end user systems that needs to be updated when the root certificate bundle changes for any other reason.

These kinds of problems are technically solvable, but they're not fun to solve, and the folks in a position to help solve them already have a great many other demands on their time.This means we're not likely to see much in the way of progress in this area as long as most of the CPython and requests developers are pursuing their upstream contributions as a spare time activity, rather than as something they're specifically employed to do.

Read the whole story
pfctdayelise
15 days ago
reply
Melbourne, Australia
Share this story
Delete
Next Page of Stories