Question

there are five red balls, three yellow balls, and four green balls in a bin. In each event, you pick one ball from the bin and observe the color of the ball. The balls are only distinguishable with their colors. After observation, you put the ball back into the bin. What is the information entropy for one event (pick ball once) in bits?

Answer 1

Answer:

Probability problem: A large number of balls, a few each of a large number of colors, are to to be cast into a similarly large number of bins. What is the expected number of distinct colors among those balls that have bins all to the themselves?

Become a data scientist: The hottest job of the 21st century.

Let X be the number of colors for which at least one ball of that color is not sharing its bin with any other ball. We want to find its expectation E(X) . To do so, for each color i∈{1,2,…,N} , we define Xi as follows:

Xi=1 if there is one ball of color i that is not sharing its bin with any other ball, and Xi=0 otherwise.

Notice that X=∑Ni=1Xi . By linearity of expectation,

E(X)=∑i=1NE(Xi)

Since the number of balls of each color i is same and equal to C , so by symmetry,

E(X)=NE(X1)(1)

Now we just need to find E(X1) which is the probability that there is at least one ball of color 1 that is not sharing its bin with any other ball. To find this, define for each ball j∈{1,2,…,C} of color 1, event

Aj= Ball # j of color 1 is not sharing its bin with any other ball.

Now,

E(X1)=Pr(X1=1)=Pr(⋃j=1CAj)(2)

By inclusion-exclusion principle,

Pr(⋃j=1CAj)

=∑j=1CPr(Aj)−∑j<kPr(Aj∩Ak)+∑j<k<lPr(Aj∩Ak∩Al)−⋯+(−1)C+1Pr(⋂j=1CAj)

By symmetry,

Pr(⋃j=1CAj)

=(C1)Pr(A1)−(C2)Pr(A1∩A2)+(C3)Pr(A1∩A2∩A3)−⋯+(−1)C+1Pr(⋂j=1CAj)

Answer 2

Only thing I read was red yellow .... and the colors so I think it’s related to probability