Question:
Consider the distribution of outcomes generated by \(N\) (\(N \gg 1\)) independent throws of (i) a coin or (ii) a six-sided dice. A coin (dice) is unbiased if both (all) its sides have equal probability to show up in a throw; it is biased otherwise. For the cases (i) and (ii) above, which of the following statements is/are true?
Consider the distribution of outcomes generated by \(N\) (\(N \gg 1\)) independent throws of (i) a coin or (ii) a six-sided dice. A coin (dice) is unbiased if both (all) its sides have equal probability to show up in a throw; it is biased otherwise. For the cases (i) and (ii) above, which of the following statements is/are true?
Show Hint
Entropy of \(k\) equally likely outcomes is \(\log k\), and this is the maximum possible entropy for that many outcomes.
Compare \(\log 2\) (coin) with \(\log 6\) (dice), and remember any bias away from equal probabilities can only lower the entropy.
Compare \(\log 2\) (coin) with \(\log 6\) (dice), and remember any bias away from equal probabilities can only lower the entropy.
Updated On: Jul 28, 2026
- The entropy of an unbiased coin is smaller than that of an unbiased dice.
- The entropy of an unbiased coin is greater than that of an unbiased dice.
- The entropy of a biased dice is smaller than that of an unbiased dice.
- The entropy of a biased coin is greater than that of an unbiased coin.
Show Solution
Verified By Collegedunia
The Correct Option is A, C
Solution and Explanation
Step 1: Recall the definition of entropy for a single throw.
For a random outcome with \(k\) possible results, each with probability \(p_i\), the Shannon entropy per throw is \(S = -\sum_{i=1}^{k} p_i \log p_i\). This number measures how spread out the outcome is: a sure outcome has zero entropy, and entropy grows as the outcomes become more evenly spread.
Step 2: Recall which distribution gives the largest entropy.
Among all probability distributions on \(k\) fixed outcomes, the entropy is maximum exactly when every outcome is equally likely, that is \(p_i = 1/k\) for all \(i\), giving \(S_{max} = \log k\). Moving away from equal probabilities, biasing the coin or dice, can only lower the entropy below this maximum, never raise it.
Step 3: Compare the unbiased coin and unbiased dice.
An unbiased coin has \(k=2\) equally likely outcomes, so its entropy is \(\log 2\). An unbiased dice has \(k=6\) equally likely outcomes, so its entropy is \(\log 6\). Since \(\log 6 > \log 2\), the dice carries more uncertainty per throw than the coin.
Step 4: Check statement (A).
From Step 3, the entropy of the unbiased coin (\(\log 2\)) is smaller than that of the unbiased dice (\(\log 6\)). So (A) is TRUE.
Step 5: Check statement (B).
Statement (B) claims the opposite ordering. Since \(\log 2 < \log 6\), this is FALSE.
Step 6: Check statement (C).
By Step 2, the unbiased dice sits at the maximum possible entropy \(\log 6\) among all dice distributions. Any biased dice has at least one face more likely than the others, moving it away from equal probability, so its entropy must be strictly smaller than \(\log 6\). So (C) is TRUE.
Step 7: Check statement (D).
By the same reasoning, the unbiased coin already sits at the maximum entropy \(\log 2\) among all coin distributions. A biased coin moves away from equal probabilities, so its entropy must be strictly smaller, not greater, than the unbiased coin. So (D) is FALSE.
Final Answer:
Only (A) and (C) are guaranteed by the fact that uniform probabilities give the maximum entropy. \[ \boxed{\text{A, C}} \]
For a random outcome with \(k\) possible results, each with probability \(p_i\), the Shannon entropy per throw is \(S = -\sum_{i=1}^{k} p_i \log p_i\). This number measures how spread out the outcome is: a sure outcome has zero entropy, and entropy grows as the outcomes become more evenly spread.
Step 2: Recall which distribution gives the largest entropy.
Among all probability distributions on \(k\) fixed outcomes, the entropy is maximum exactly when every outcome is equally likely, that is \(p_i = 1/k\) for all \(i\), giving \(S_{max} = \log k\). Moving away from equal probabilities, biasing the coin or dice, can only lower the entropy below this maximum, never raise it.
Step 3: Compare the unbiased coin and unbiased dice.
An unbiased coin has \(k=2\) equally likely outcomes, so its entropy is \(\log 2\). An unbiased dice has \(k=6\) equally likely outcomes, so its entropy is \(\log 6\). Since \(\log 6 > \log 2\), the dice carries more uncertainty per throw than the coin.
Step 4: Check statement (A).
From Step 3, the entropy of the unbiased coin (\(\log 2\)) is smaller than that of the unbiased dice (\(\log 6\)). So (A) is TRUE.
Step 5: Check statement (B).
Statement (B) claims the opposite ordering. Since \(\log 2 < \log 6\), this is FALSE.
Step 6: Check statement (C).
By Step 2, the unbiased dice sits at the maximum possible entropy \(\log 6\) among all dice distributions. Any biased dice has at least one face more likely than the others, moving it away from equal probability, so its entropy must be strictly smaller than \(\log 6\). So (C) is TRUE.
Step 7: Check statement (D).
By the same reasoning, the unbiased coin already sits at the maximum entropy \(\log 2\) among all coin distributions. A biased coin moves away from equal probabilities, so its entropy must be strictly smaller, not greater, than the unbiased coin. So (D) is FALSE.
Final Answer:
Only (A) and (C) are guaranteed by the fact that uniform probabilities give the maximum entropy. \[ \boxed{\text{A, C}} \]
Was this answer helpful?
0
0
Top GATE PH Physics Questions
- An infinite one-dimensional lattice extends along the x-axis. At each lattice site, there exists an ion with spin 1/2. The spin can point either in the +z or -z direction only. Let \( S_P \), \( S_F \), and \( S_A \) denote the entropies of paramagnetic, ferromagnetic, and antiferromagnetic configurations, respectively. Which of the following relations is/are true?
- Two point charges of charge \( +q \) each are placed a distance \( 2d \) apart. A grounded solid conducting sphere of radius \( a \) is placed midway between them. Assume \( a^2 \ll d^2 \). Which of the following statements is/are true?
- A particle of mass $m$ is moving in the potential
\[ V(x) = \begin{cases} V_0 + \frac{1}{2} m \omega_{0P}^2 x^2 & \quad \text{if } x > 0 \\ \infty & \quad \text{if } x \leq 0 \end{cases} \]
Figures P, Q, R, and S show different combinations of the values of $ \omega_0 $ and $ V_0 $. - Decays of mesons and baryons can be categorized as weak, strong and electromagnetic decays depending upon the interactions involved in the processes. Which of the following option is/are true?
- The temperature \( T \) dependence of magnetic susceptibility \( \chi \) (Column I) of certain magnetic materials (Column II) are given below. Which of the following options is/are correct?
Column I Column II (1) 
(P) Diamagnetic (2) 
(Q) Paramagnetic (3) 
(R) Ferromagnetic (4) 
(S) Antiferromagnetic
View More Questions
Top GATE PH Questions
- If \( F_1(Q, q) = Qq \) is the generating function of a canonical transformation from \((p, q)\) to \((P, Q)\), then which one of the following relations is correct?
- When an unpolarized plane electromagnetic wave in a dielectric medium with refractive index \( n_1 \) is incident on a plane interface separating it from another dielectric medium with refractive index \( n_2 \) (where \( n_2 > n_1 \)), and the angle of incidence is \(\tan^{-1} \left( \frac{n_2}{n_1} \right) \), the following statement is true:
- The wavefunction of a particle in an infinite one-dimensional potential well at time \( t \) is
\[ \Psi(x, t) = \sqrt{\frac{2}{3}} e^{-iE_1 t/\hbar}\psi_1(x) + \frac{1}{\sqrt{6}} e^{i\pi/6} e^{-iE_2 t/\hbar} \psi_2(x) + \frac{1}{\sqrt{6}} e^{i\pi/4} e^{-iE_3 t/\hbar} \psi_3(x) \]where \(\psi_1\), \(\psi_2\), and \(\psi_3\) are the normalized ground state, the normalized first excited state, and the normalized second excited state, respectively. \(E_1\), \(E_2\), and \(E_3\) are the eigen-energies corresponding to \(\psi_1\), \(\psi_2\), and \(\psi_3\), respectively. The expectation value of energy of the particle in state \(\Psi(x,t)\) is - If a thermodynamical system is adiabatically isolated and experiences a change in volume under an externally applied constant pressure, then the thermodynamical potential minimized at equilibrium is the
- The mean distance between the two atoms of an HD molecule is \( r \), where H and D denote hydrogen and deuterium, respectively. The mass of the hydrogen atom is \( m_H \). The energy difference between the two lowest lying rotational states of HD in multiples of \(\frac{h^2}{m_H r^2}\) is:
View More Questions