Tag: BS

Bear Stearns

Simple Sample SAML Service Provider Programmed in Python

Try repeating the title of this blog posting three times fast.
And if you think that is tough, try coding one. I haven’t been paid to write computer programs since I left the research faculty of the Albert Einstein College of Medicine back in 1986, so my coding skills could be a little rusty, although I have written lots of simple scripts, modified dozens of others in multiple languages and help programmers debug their programs in more languages than I can count, but more about that in another post. This one is a rant about Python.

I was first introduced to Python as a programming language in 2001, when my boss at Bear Stearns at the time (who shall remain nameless) promised the London research team that he would speed up the delivery of their research Emails. At the time we were using Sendmail with some intervening shell scripts to manage our outbound Emails and the London research team was sending their research out to lists of several hundred addresses. Since Sendmail in those days was single-threaded the first addresses in the list got the research fairly quickly (by the standards of those days). Unfortunately the owners of the lists didn’t always keep them clean and up-to-date, so there were often bad addresses or addresses with misspelled domains. Since Sendmail would try every MX server for a domain, or the A record for the domain if there was no MX records, then every bad address or domain would slow down the delivery to subsequent recipient addresses. By the time Sendmail got to the end of the list the timely research was often obsolete so our London research team, and their customers, were understandably upset. My boss’ solution was to obtain 2 Unix servers, powerful Solaris boxes at the time, download Postfix, Python and Mailman onto those servers, and then hand them over to me and resign from the firm. It became my job to put all this together so that the London research team, and ultimately several other Bear Stearns teams, could use these servers to send out their research in a timely manner.

Mailman, for those of you who don’t know it, is open-source mailing list manager software written in Python. The then current version of Mailman did not include “Real name” support for members which I see is now a feature of the current version, but our users required it, since they couldn’t be bothered knowing the actual Email addresses of their clients. That version also didn’t include the concept of a list member manager separate from the list manager, although we wanted our research people to be able to maintain their mailing lists without having any access to the other features of their mailing lists. Thus, I had to write an entire new user interface for the Mailman mailing lists which allowed the list owners to import/add/delete real names and Email addresses for their clients but which hid from them the other features of Mailman and their lists. Fortunately Python is an object-oriented language and the Mailman lists were nested objects so it was not too difficult to add attributes to the list objects for real names, and to modify the user interface to restrict what our list owners could do. Of course, first I had to teach myself enough Python to understand the Mailman source code and figure out how to modify it. That took a couple of weeks. As I recall the Mailman code was written in Python 1.5, so things have changed a lot since then, but that was my introduction to Python.

Fast forward to 2019, where I’m helping an old friend from BS with some software she is writing in Python and she determines that she needs to be able to do Single Sign On (SSO) using SAML for one of her customers. This being the era of Linux, open-source software and shared library modules I searched for a Python module that could be used as a SAML Service Provider. I found a few, but none had adequate documentation to just plug them in and most were designed for specific web frameworks. My friend was writing her code in Python 2, using a web framework written by another old BS alumnus which mostly outputs JSON and was unable to supply the 302 status which the browser needed for simple HTTP redirects to the SAML IdP. Also, this being 2019 and the last year that Python 2 will be supported (although I see that there are still some utilities which may not be Python 3 ready), my code had to work with Python 2, but be upward compatible to Python 3. I managed to get a working proof of concept (POC) for the code using Apache and Python 2 CGI, but it is still clunky.

Moving the code from Python 2 to Python 3 has been more of a headache than anticipated, mostly because of the change in the way strings are handled. Distinguishing between byte strings and Unicode strings is very necessary, but it becomes a pain to manage when modifying lots of legacy code. But that’s not my major complaint about Python. Maybe my complaint is just because I haven’t taken the time to understand the issues involved, but it seems that the method Python uses for locating system modules has evolved over the last 20 years in not always compatible ways. The latest idea, of every application having its own environment with its own set of library modules may make sense in these days of really cheap memory and storage, but is difficult for us old-timers who are used to having limited memory to work with. Here again I will save this for another post.

I’ve had several utilities which are coded in Python and which self-update, but which have been unable to find their modules since the default Python was modified from Python 2 to Python 3, even if they have their own version of Python in their environments. I’m not sure how to tell these utilities how to find the commonly installed modules, or how to install needed modules into their specific environments. I’m sure I will figure this out in the next day or two, but it would have been great if Python was able to do it by itself without forcing me to go through these contortions to make things work.

Enough minor ranting for now, but I did make some promises above for more posts in the future. I’m hoping to put together more, shorter posts. We’ll see if I can do that.
Thanks for reading this.

DNS at Bear Stearns

In my last post I mentioned “DecNet Terminal Servers” which connected the dumb terminals to the big Vaxen in the Whippany datacenter. Those terminal servers actually used LAT, which was a Dec proprietary protocol to connect the dumb terminals to the Vaxen. Shortly after I got to Bear Stearns someone in the purchasing department discovered that they could purchase terminal servers which spoke LAT for less than the cost of the Dec terminal servers. These other terminal servers also spoke telnet over TCP/IP, so that the same dumb terminals could be used to connect to the Amdahls and Sun Microsystems servers. Unfortunately those terminal servers needed DNS to resolve the names to IP addresses and there weren’t any DNS servers at Bear Stearns. At the time, the Unix boxes were using NIS, which was then known as Sun’s Yellow Pages (YP).

I was able to get a TCP/IP stack for VMS from a company called TGV (Two Guys and a Vax, not to be confused with the French high speed railroad). In addition to the TCP/IP stack TGV also supplied versions of standard Unix utilities like Bind (the standard DNS server) and Sendmail (the standard SMTP server). Sendmail allowed me to connect the VMS Email system to Sun Unix Email, cc:Mail on the PCs and Profs on the mainframes. Bind let me run a DNS server on one of our spare VaxStations. The issue was creating the zone files (the lists which relate the host names to their IP addresses). There were hundreds of hosts and maintaining the Bind zone files by hand was not feasible. Back in the days before the Internet, people shared information and programs via Usenet over UUCP. As I mentioned in my first post, Email in those days was delivered overnight. I won’t go into details about UUCP and Usenet in this post, but Wikipedia has articles on many of these topics. Usenet newsgroups were also transmitted via UUCP overnight and they allowed people to share information about programs and other issues. When I went looking for a program to convert Unix host files into Bind zone files I discovered that there was a program named h2n which consisted of several thousand lines of AWK code but was very flexible. To get the program I had to send an Email to several FTPmail servers asking for a list of available programs to see which server had h2n. Then I had to send an Email to the proper FTPmail server asking it to send me the program. In those days, programs came as several Email messages which had to be concatenated together to make a shell script which created the file (or files) needed to run the program. All in all it took several days to get the program. Once I had the program I set up a scheduled job to run ypcat to save a copy of the NIS (YP) hosts map to a file, then I ran h2n against that hosts file to create the needed forward (name to IP address) and reverse (IP address to name) zone files. Those files were then copied to a VMS VaxStation (fixt31 sitting under my desk) which was running Bind and that was used by the new terminal servers to map the names to IP addresses. That was the first DNS server at Bear Stearns, and remained the master DNS server for many years.

Enough for today, I hope to get another post out in the next week or two. That one will be a break from my Bear Stearns past. I hope to write about a recent project I was working on.

More early BS (Bear Stearns) memories…

But first, Holly suggested I left out two pieces of information from the last posting so I’ll give you those. The first is how to say the name of this blog. In ‘techspeak’ the exclamation mark is called a ‘bang’, so my early Email address at Bear Stearns, as well as this blog, is sol bang ursa bang ari. The second piece of information that Holly said I should include is for those of you who have forgotten your Latin or your constellations. Ursa is Latin for bear, so the computer which was the UUCP gateway to Bear Stearns was called bear.

Speaking of Bear Stearns, back in 1988 when the head-hunter told me he was sending me for an interview at Bear Stearns I thought it was a streaking club. If you don’t get that, ask in the comments and I or someone else will explain it.

I was hired by the IT department at Bear Stearns to support the Fixed Income trading floor, specifically Pete the new head of mortgage research. My initial assignment was to install a network of VaxStations running VMS for the traders, research analysts and developers. There were already a few VaxStations around the 4th (FI trading) and 5th (FI research) floors at 245 Park Ave., but most of the people had dumb DEC (VT320 or color VT340) terminals connected via DecNet Terminal Servers to the big Vaxen in the Whippany datacenter, along with several smaller green Quotron screens. On my first day on the job I found about 15 VaxStation 3200’s still on their pallets on the 5th floor, waiting for me to unpack them and install them around the group. The VaxStation 3200’s were desk-side units, about the size of a radiator in a typical NYC pre-war apartment, so they were difficult to install on the crowded 4th floor trading desks, but we managed to fit a few in, along with their 19 inch color CRT monitors which weighed about 70 pounds. DEC soon came out with their VaxStation 3100 model, which was a desktop model more the size of a pizza box, so we ordered lots of those. Pete didn’t want his people to be able to take their software or data home with them on the 3-1/2 inch floppy disks which came standard with the VaxStation 3100’s, but DEC didn’t sell a preconfigured 3100 without the floppy drive, so he ordered diskless computers and then ordered SCSI controllers and 2 disks for each computer. That meant that I had to install the hardware and the software on each of the stations before it could be used. The 3100’s all came with thin-wire Ethernet, basically coaxial cable BNC connectors which allowed them to be easily networked together. They also each came with a phillips-head screwdriver for assembly. I was able to install the controller cards and disks into the servers, then daisy-chain them together with the coaxial cable and connect them to one VaxStation which had the software installed. I wrote some scripts to automatically install the software, setting the DecNet address for each computer based on the MAC address of the Ethernet interface so that they each got unique address and since the boxes had the MAC addresses I was able to give each one a unique name tied to the address. All of this is standard now, but in those days it was a fairly new idea. I set up the configuration table of MAC addresses and names, then started the script and left for the night. The next morning I would come in to a half-dozen new workstations ready to be installed on the desks. I used a simple naming convention based on the use of the computer and a sequence number, so the ones I installed were called boring names like fixd02 for a developer workstation or fixt31 for a trader workstation. The earlier VaxStations had more interesting names. The ones running VMS were named for denizens of the sea (fluke, mako, squid, etc.) while the ones running Ultrix (DEC’s proprietary Unix variant) were named for birds who ate denizens of the sea (osprey, heron, etc.). Interestingly enough, the Ultrix group eventually gave up and those boxes were reimaged with VMS, so the sea denizens ended up eating their predators.

Shortly before I got to 245 Park Ave. Bear Stearns’ networking group had started to run fiber to the trading desks so each workstation that I installed had to have a fiber to Ethernet transceiver attached. In those days, the standard Ethernet connector, when not using the thin-wire BNC connectors, was a 15 pin slide-lock connector, but Bear Stearns decided to replace all of the slide-locks with screw-down connectors because of a mishap with a Sun Workstation on the equities trading desk. I only heard about that third-hand so I won’t go into it here.

There’s lots more from these early days, some of which I still remember, but I’ll save more for another post. I hope this didn’t bore too many of you too much.

Introduction to my blog: sol!ursa!ari

Welcome to my blog. I’ll start with an explanation of the name of the blog. That was my Email address when I first joined Bear Stearns back in the late 1980’s. Yes, that was before most people knew what Email was, and before most people consider the Internet to exist. Back in those days Email was delivered from computer to computer via UUCP (Unix to Unix CoPy) over the POTS (Plain Old Telephone Service) or standard dial-up phone lines, mostly at night when the long-distance costs were less. (Yes, in those old days telephone calls were charged for by time of day, length of call and distance from caller to receiver). Each computer only contacted a small number of other computers, so you needed a path from your computer to the recipient’s computer when sending Email. To simplify the process there was a UUCP mapping project which collected the list of computers and which other computers they exchanged Email with. Using the path mapping information and programs like pathalias you could find the path from your computer to any other computer which was listed and then create the path. In the case of sol!ursa!ari, sol was a well-known computer at Columbia University which our computer at Bear Stearns (ursa) called nightly to exchange Emails. Thus, I only needed to start my Email path with sol for most others to be able to find me. I was the user ari on the computer ursa which exchanged Email with the well-known computer sol.

More about this, and lots of other things, in future posts.