The concept of a lich's phylactery is taken from Judaic mysticism. In reality, phylacteries were a complex kind of 'magic underwear' which were apparently quite common in Jewish communities at one time. Jewish Encyclopedia.com has an absolutely fascinating article on the subject, written in the early 20th century. There's an impressive amount of detail there, much of which I think I would need to know a lot more about Jewish tradition to fully understand. But enough of the article is written in plain English for me to learn a lot about the beliefs surrounding this tradition.
As I mentioned in my post titled Succubi Deserve More, I like to explore the mythology behind fantasy tropes. Not only does it result in me becoming a more educated and historically aware person, but the real-world mythology always offers fascinating insight into the fantastic possibilities. Whoever first decides to take some cultural or mythological element and include it in a fantasy story takes what works for them, and leaves the rest. That's how fantasy writing works. But who is to say that the elements they left behind aren't sometimes just as interesting as the elements they chose to keep?
For clarity's sake, lets start with the explanation of what a phylactery is in Pathfinder, pulled from The Pathfinder Bestiary, page 188. For those curious, this excerpt is functionally identical to the same excerpt in the Dungeons and Dragons 3.5 Monster Manual.
An integral part of becoming a lich is the creation of the phylactery in which the character stores his soul. The only way to get rid of a lich for sure is to destroy its phylactery. Unless its phylactery is located and destroyed, a lich can rejuvenate after it is killed. (See Creating a Lich, below).
Each lich must create its own phylactery by using the Craft Wondrous Item feat. The character must be able to cast spells and have a caster level of 11th or higher. The phylactery costs 120,000 gp to create and has a caster level equal to that of its creator at the time of creation.
The most common form of phylactery is a sealed metal box containing strips of parchment on which magical phrases have been transcribed. The box is Tiny and has 40 hit points, hardness 20, and a break DC of 40.
Other forms of phylacteries can exist, such as rings, amulets, or similar items.
Not a lot to go on, really. I also seem to recall very distinctly that the process of becoming a lich (and so, presumably, creating the phylactery) is supposed to be profoundly evil. To my knowledge, that is the sum of official material on what a phylactery is within the game world. There are probably a few dragon magazine articles, and sourcebooks from the 70s and 80s which contain further tidbits of "official" information, but for now the basic definition will do.
Before moving any further, I would like to again remind my readers that I am not a credible source on the topic of Judaic history and lore. The sources for this post, which have far more information on this topic, are the Jewish Encyclopedia.com article on Phylacteries, and the Skeptic's Annotated Bible.
The historical phylactery, by comparison, was considered a very holy thing. In fact, if you look at the word's etymology, the Greek root words suggest that it was intended to protect the wearer from evil. The Jewish custom is based on a number of passages in the Torah, most notably this excerpt from Deuteronomy:
And these words, which I command thee this day, shall be in thine heart: And thou shalt teach them diligently unto thy children, and shalt talk of them when thou sittest in thine house, and when thou walkest by the way, and when thou liest down, and when thou risest up. And thou shalt bind them for a sign upon thine hand, and they shall be as frontlets between thine eyes. And thou shalt write them upon the posts of thy house, and on they gates.
The 'words' this passage wants the reader to spend so much time talking about are, as best I can determine, God's laws. Variations of this passage show up in a number of places, since repetition is an essential element in an oral tradition. The important part, though, is the bit I emphasized. That's the origin of the historical phylactery. The exact means of how these devices were worn is somewhat unclear to me. The image at the start of the post demonstrates how complicated they appear to be--and every element was important. Even the way the knots were tied was meant to symbolize specific Hebrew lettering. Essentially, however, historical phylacteries are small boxes or pouches which are worn on the arms and between the eyes. Within the pouches are a specific arrangement of passages from the Torah, written on tiny scrolls of paper. This is likely where the idea of a lich's phylactery being a metal box filled with tiny magical scrolls came from.
One of the archetypical things which liches do is hide their phylacteries. Common ideas are to hide it in a fortress somewhere, or to give it to a powerful dragon to protect. I've been involved in discussions on /tg/ and elsewhere which focus just on coming up with the most outrageous, funny, and clever ways to hid a phylactery. And I've heard some positively fantastic ideas. But the historical phylactery was a thing which had to be worn. You couldn't leave it at home and continue to rely on the spiritual protection it provided.
Of course, if every lich was wearing their phylactery dangling between their eyes, the monster would loose all of its flavor. But what if there was a limit to how far away the phylactery could be from the lich? Say, it must be within 1 mile of the lich's location. For each additional mile away, the lich suffers from 1 negative level, and if the lich reaches 0, it dies and re-forms at the location of its phylactery. Perhaps the lich might even get some kind of bonus if its phylactery is within 100ft, say, plus one caster level? Adding a mechanic like this takes nothing away from the the fun of hiding the lich's phylactery, and in fact may end up being a great deal more fun for the players. Looking for a hidden item can be fun, but if that item is in an adamantite box which shifts to a random location in the multiverse every 30 seconds, the players are simply going to get bored. Adding limits gives the players somewhere to start their investigation. Plus, this adds a fun element to the game of a lich needing to actively manage their phylactery's location in order to avoid negative levels.
Also interesting is that the wearer of a historical phylactery was not supposed to enter a cemetery, or "any unseemly places" whilst wearing it. Again, this suggests some interesting possibilities for the lich's phylactery. Since liches never have their phylactery, it wouldn't make sense for certain places to only be accessible when the lich didn't have it, but what if there were certain places a lich couldn't enter UNLESS it had its phylactery with it? Such as an area which is consecrated, or perhaps they cannot go within 10 miles of their original birthplace without their phylactery. It might even be interesting to say that a lich could never enter a cemetery without its phylactery. Though, given a lich's frequent need for necromancy reagents, this could make things difficult.
There are a number of rules for historical phylacteries...actually there are a plethora of rules. There is an entire pantheon of rules. This is, after all, Judaism. The rules range from the spacing on the letters on the little scrolls, to the attention span of the chap scribing those letters, to even the color of the case. Largely, I don't think these have much application. They could be fun if one was trying to come up with a good ritual for creating a phylactery, but unless a character becoming a lich is the focus of a campaign, I don't think it's particularly useful to go into the creation process too much. Although that would be a kickass campaign.
However, this rule caught my eye: "The straps (Yad. iii. 3) were made of the same material as the boxes, but could be of any color except blood-red." Perhaps I'm shooting in the dark, here, but what if blood were harmful to phylacteries? What if, perhaps, blood was the ONLY thing which could harm a phylactery. The blood of a goodly person--or perhaps even the blood of a fallen hero. The phylactery must be coated with it, and then it becomes as brittle as a twig.
I encourage you to read up on the historical phylactery yourself, and comment on your own ideas for making a lich's phylactery more interesting!