pyspark.sql.functions.approx_count_distinct

pyspark.sql.functions.approx_count_distinct(col, rsd=None)

Aggregate function: returns a new Column for approximate distinct count of column col.

New in version 2.1.0.

Parameters:

colColumn or str

rsdfloat, optional

maximum relative standard deviation allowed (default = 0.05). For rsd < 0.01, it is more efficient to use countDistinct()

Examples

>>> df.agg(approx_count_distinct(df.age).alias('distinct_ages')).collect()
[Row(distinct_ages=2)]

pyspark.sql.functions.approxCountDistinct pyspark.sql.functions.array