REVISED ANSWER
- First Approach, from Random Graphs
Never seen it anywhere. Anyway, here is my two-cents:
First of all, there is a well established theory of RANDOM GRAPHS (see here for a ref. Of course, there are many choices available, you may make nodes, edges or both random, and you may confine yourself to certain classes of probability distributions).
Now, a a category is a (directed) graph, in the precise sense that there is an obvious forgetful functor $ F: Cat \rightarrow Graph$.
The first step toward defining a random category would be to say that it "forgets" to a random graph. In other words, just like the category of (deterministic) Graphs embeds as a subcategory of Random Graphs, the category of standard deterministic categories CAT would embed in the cat of Random Categories, in such as way as to preserve the respective forgetful functors.
A category has more structure than a directed graph, it has composition, identity, and also commuting diagrams, so you need to add those to the picture:
composition is easily taken care of, for instance by stipulating that the probability of a composite arrow is the product of the probability of its components. Similarly, identity morphisms could be assigned the same probability of their corresponding nodes.
Now you are left with the diagrams (I am talking now of only those diagrams which are not forced by the laws of composition). There you have a certain degree of freedom. My intuition would be: first play with random free categories constructed from random graphs by free completion, then stipulate an assignment of probabilities to the space of all generating diagrams in the category.
- Second Approach, from Random Simplicial Sets.
Another (broader) take on the above: categories are, in a sense, particularly simple types of SIMPLICIAL SETS, namely the ones which satisfy the horn condition.
So, rather than starting from random graphs and building your way up to categories, start with a notion of random simplicial set and by restriction you will have your random categories.
How? Don't know if the notion of random simplicial sets has been explored, but certainly there is something on random simplicial complexes, see for instance here). This could be used as motivating example.
Hope it helps
PS Final comment: in fact approaches 1 and 2 are in the end the same, 1 is part of 2, because graphs are simplicial sets of dimension 1.