Iconicity, 2^nd‐order isomorphism, and perceptual categorization

Ned Block's project is to articulate and defend a border between perception and cognition by drawing upon advances in perception science. He draws upon perception science, first, in identifying empirical signatures of perception (adaptation, rivalry, pop-out, and illusory contours), and, second, in proposing more fundamental features of perception (non-conceptuality, non-propositionality, and iconicity). The more fundamental features would explain why the signatures tend to cluster as they do and thus constitute perception as a natural kind. My comments focus on iconicity as a fundamental, constitutive feature of perception. Block's conception of iconicity requires that perceptual representations and what they represent come in degrees (or, are graded, as I'll also put it). But there are apparent cases of perception that are not obviously iconic in this sense—for example, perceptual categorizations. Various readers of Block's book have independently noted this prima facie tension, including Green (2023), Firestone and Phillips (2023), and Beck (2023). This commentary develops the issue and explores possible replies. The replies differ in their burdens and commitments, or what might be costs. I look forward to learning which, if any, Block might favor. Iconic representations are often said to be picture-like or image-like. But in what sense? There are various conceptions of iconicity in the literature. It's useful to put to one side a prominent conception that Block does not prefer. On this view (refinements aside), a representation is iconic just in case parts of the representation represent a part of the what the whole representation represents.1 For example, if you photograph a scene and cut out a section of the photograph, generally the part you've cut out itself represents part of the scene. Not so for such non-iconic representations as sentence-tokens. Suppose that firing rates of certain neural circuits represent time duration. Although one duration has another duration as a part, a firing rate of 50 times per second does not have a firing rate of 17 times per second as a part. (Block, 2023a, p. 225)2 Here's Block's full statement of his “analog tracking and mirroring” conception of iconicity: Analog tracking and mirroring obtains when there is a set of environmental properties and a set of representations of those environmental properties such that: Certain differences in representations function as responses to differences in environmental properties in a way that is sensitive to the degree of environmental differences. … Certain differences in representations function to alter the situation that is represented in a way that depends on the degree of representational change. Certain relations (including temporal relations) among the environmental properties are mirrored by representations that instantiate analogs of those relations. (Block, 2023a, pp. 221–2) Crucially for our discussion, iconic representations on Block's view represent properties that come in degrees, and they do so by coming in families of representations themselves organized by degree of vehicular change. Degree of change in representational content corresponds (as a matter of function) to degree of change in representational vehicle and vice versa. Indeed, Block emphasizes that “[t]he key difference [between iconic and discursive representations] comes in with the role of degrees of difference” (Block, 2023a, p. 222—italics in original). Iconic representation, unlike non-iconic representation, thus requires a family of properties organized along some graded dimension(s) and a family of representational vehicles likewise arrayed along some graded dimension(s) such that the placement of features and vehicles, respectively, along these dimensions relevantly mirror one another. Peacocke's counter-example to the Parts Principle provides an illustration: The loudness of sounds comes in degrees, and iconic representations as of loudness in principle might be realized in a correspondingly graded way by average number of spikes by some neural population across a temporal interval: the more spikes, the louder the sound is represented to be. Different spike rates would correspond to different volumes, and the representations would be organized along a dimension—number of spikes per unit time—that mirrors the greater or lower volumes of the sounds.3 What is meant here by coming in degrees? Block leaves the notion undeveloped, beyond noting that density is not required and magnitudes can be “digitized” (his quotes—Block, 2023a, p. 222). Rather than attempt an explication on his behalf, I'll hope that any mistaken understanding on my part will occasion useful clarification. But I will take seriously the idea that the relevant environmental properties are in some sense degreed and that an accurate perceptual attribution of these properties must in some sense capture this and do so in part in virtue of degreed aspects of the representational vehicle, as per clauses (1) and (2) of Block's characterization of iconicity. (As Block (2023a, p. 222) notes, clause (3) also involves degrees: the “mirroring of degrees by degrees”.) In section 4, I'll consider a weakening. The prima facie problem is that there seem to be perceptual attributions that are not iconic in Block's sense. For example, Block, in agreement with many others, maintains that we perceptually attribute being a face, being a certain phoneme, being an object (or being a Spelke-object), being inside of, being the cause of, etc. But, in each case, it's not obvious that the requirement of gradedness is satisfied. I focus on perceptual categorization. Perceptual categorizations are perceptual attributions as of belonging to a certain category. The term ‘category’ gets explicated in various ways in these contexts. For example, Burge (2022, p. 487) limits perceptual categorization to attribution of kinds, while Block (2023a, p. 271) includes color categorizations—attributions of features, not kinds, on Burge's view. What matters for us is just whether the attributions are iconic in Block's sense, however they are themselves categorized. Again, a reason for thinking they are not iconic is that they seem to attribute non-graded attributes. Some actually build non-gradedness into their characterization of categorization—e.g., Mather (2016, p. 143): “Each stimulus is perceived as a member of a discrete category, in an all or nothing fashion, rather than occupying a position along a continuous dimension.” But it's more informative to explore arguments for the claim, rather than assume it from the start. A first argument is that certain ways of speaking or thinking that would indicate gradedness seem unacceptable. For example, it seems unacceptable to speak or think of one thing as more of a face, or “facier”, than another.4 Such considerations are indeed a common source of evidence in linguistic semantics—for example, in discussions of graded adjectives (Kennedy and McNally, 2005). But, as this very fact reminds us, they fall short of establishing ungradedness in perceptual attribution. The unacceptability judgments may just reflect features of our corresponding concepts and/or the lexical items that express them. That a conceptual attribution, or its linguistic expression, does not attribute a graded property does not entail the same of a corresponding perceptual attribution. What could follow—depending on how Block might develop his talk of degrees—is that what's attributed in perception is not the same as what's attributed conceptually in such cases. This would be so, for example, if degreed attributions attribute degreed properties, but non-degreed attributions don't: one and the same property can't be both graded and ungraded. Or if, say, what degreed attributions attribute are relations between a particular and position on a scale, while what non-degreed attributions attribute are monadic. The transition from the perceptual attribution to the corresponding conceptual attribution—a transition perhaps mediated by a graded cognitive attribution—would then involve in that respect a change of content. Moreover, when graded perceptual attributions were characterized using ungraded language (for example, perceptual attribution as of being a face, sans degree), we would want to understand this as a façon de parler that exploits a ready-to-hand lexical expression to indirectly indicate its corresponding perceptual attributive. A second way to argue for non-graded perceptual attribution adverts to empirical demonstrations of categorical perception in the technical sense. Categorical perception in the technical sense is when equidistant items or features along a certain objective dimension are perceived as more similar within certain boundaries than across them. These boundaries are then theorized to be the boundaries of categories. For example, one can vary relevant aspects of the acoustic signal continuously, but have subjects judge language-like sounds within boundaries to be more similar than objectively equidistant sounds across boundaries. In part on this basis, it's claimed that subjects perceive phonemes in a way that conforms to there being fairly sharp boundaries between them. But does categorical perception in fact challenge Block's claim that perceptions are constitutively iconic in his sense? Categorical perception is often described as a warping of the represented feature space relative to how the features are related objectively: some parts of the space are stretched, others smushed, relative to the objective space (Goldstone and Hendrickson, 2010; Kronrod et al., 2016). For example, color hue may vary continuously, and our perceptual representations of color may be graded (down to the limits of acuity); but, if there is categorical perception of color (an unsettled question—cf. Witzel, 2019, also McMurray, 2022 on phonemes),5 the “distance” between color hues in the represented space may be deformed in a way that leads to the judgments characteristic of categorical perception. If so, further argument, beyond the existence of categorical perception in this sense, would be needed to conclude that perceptual representation of colors isn't graded. Moreover, it's worth noting that the deformations in this case would preserve ordinal relations among colors, satisfying clause (3)—the “mirroring” clause—in Block's characterization of iconicity. One further argument would advert to downstream effects. If representational states between “boundaries” share their downstream effects, that would provide prima facie reason to assign them the same content—for example, red as opposed to, or in addition to, more fine-grained contents corresponding to degrees of redness. Similarly, one may argue that what matters crucially downstream in linguistic comprehension (in particular, for lexical identification) is what phoneme is represented—not, or not just, the graded lower-level features in the acoustic signal on which phoneme perception in part depends in an extremely complex way (Kazanina et al., 2018). This is not the place to delve into the empirical details necessary to develop and evaluate such arguments. But I'll mention two strategies in reply. The main point is just to indicate—what's perhaps obvious—that these matters involve substantial, unobvious empirical commitments. One reply would argue that these downstream effects and, importantly, the representations responsible for them are post-perceptual. This would seem to be in tension with other of Block's commitments: Block (2023a, pp. 64–9—and cf. pp. 271–3) discusses at length the susceptibility of phonemic representations to adaptation, evidence that they are perceptually attributed. But perhaps the tension could be resolved by positing degreed perceptual phonemic attributions (susceptible to adaptation) that transition to non-graded post-perceptual phonemic attributions that are more directly responsible for the downstream effects—similarly for other cases. To be compelling, this obviously would require empirical backing. Alternatively, one might try reinterpreting the phonemic adaptation results in terms of adaptation to lower-level features (cf. Block 2023a, p. 65). The other strategy would argue that these downstream effects can result from computations over graded representations. There is indeed significant work on how to computationally relate graded and non-graded representations (e.g., Smolensky et al., 2014). Here, though, I would caution that not all such work sees graded representation as thereby attributing graded properties or otherwise including gradedness in the representation's content, as Block's clause (2) would seem to require. That a representation of /b/, for example, can be activated to varying degrees need not entail that an instance represents being /b/ to such-and-such degree (Kazanina et al., 2018). Among other possibilities, the level of activation might correspond to a perceptual credence (Feldman et al., 2009).6 We've been exploring ways of arguing that perceptual categorizations are not graded, as well as responses thereto. The discussion has put much weight on Block's talk of degrees, appropriately so given his own emphasis. But there's another line of reply worth exploring—one that just drops this talk of degrees. This might count as a reply, not a capitulation, if it would otherwise preserve what's crucial in Block's characterization of iconicity. (It would definitely count as a reply, not a capitulation, if it in fact captures what Block intends by his admittedly undeveloped talk of degrees, while just dispensing with a possibly misleading term.) The idea is that iconicity might require just that there be a family of representations arrayed along representational dimensions that mirror some environmental dimension(s)—where the representations function to be sensitive to differences along the environmental dimension(s), and changes of representational vehicle function to change representational content along the relevant dimension(s). Iconicity might require that, without requiring further that the representational changes and environmental differences come in degrees. Phonemes, for example, could find their place in a phonemic space. But it needn't be that one sound is more of a /t/ than another or that perceptual phonemic attributions represent or entail that they are—even if the sounds may come in degrees along the lower-level dimensions that constitute the space. (Compare the overlay of political boundaries on a spatial region: locations may be closer and further from one another, but either in a state or not, not to some degree.) Phonemic representations could then be iconic insofar as the representations form a space that mirrors relations among the phonemes and represent the phonemes in part in virtue of doing so—without requiring phonemes, or representations thereof, to be graded. This line of reply would have the added benefit of handling not just prima facie non-graded perceptual categorizations, but also other apparent non-graded perceptual attributions. But there's a new problem. There seem to be perceptual attributions that don't (or don't obviously) fall into a family of such representations—representational loners, if you will. Examples might be highly generic categories or features, such as being colored or shaped. Representations as of being red fall into a family with other color representations. But what of a representation as of being colored—not as of being this or that color, but simply as of being colored? (Similarly, for a representation as of being shaped.) What is the family to which it belongs, such that members of the family are arrayed in a space and it is empirically plausible that they represent similarly arrayed properties in part in virtue of falling into such a family with vehicles thus arrayed? Neither the family of most generic perceptual determinables nor representations thereof seem so arrayed. Of course, such cases pose no problem if in fact we do not make such highly generic perceptual attributions. Block may deny that we perceptually attribute being colored or being shaped—contra Burge (2022). But there are perceptual representations Block does defend which likewise may be loners. An example may be perceptual attributions as of something's being an object, or as of being a Spelke-object: a relatively rigid, cohesive 3-d body with a closed boundary (Spelke, 1990). What is the family to which perceptual attribution of objecthood belongs, such that members of the family are arrayed in a space and it is empirically plausible that they represent properties similarly arrayed in part in virtue of falling into such a family with vehicles thus arrayed? It might be tempting to reply that loners are just limit cases: a family of one kind of representation, related to itself and perhaps its absence.7 As such, they might seem to satisfy our weakened clauses for Block's conception of iconicity. Clauses (1) and (2) are satisfied insofar as a change in Spelke-objecthood tends to lead to a change in perceptual representation, and a relevant change in the perceptual representation changes whether it represents something as a Spelke-object. As for clause (3), perhaps Spelke-objects’ being of the same kind as one another is mirrored by representations thereof likewise being of the same kind. The problem, however, is that this trivializes iconicity so that even paradigmatically discursive representations count as iconic. Recall Block's underscoring how gradedness is key to distinguishing iconic and discursive representations. On our weakening, multiplicity is needed to play the same role. Here's a different reply to the prima facie problem of loners. Attributions of Spelke-objecthood are higher-level in that they depend on the attribution of other properties: cohesiveness, rigidity, and boundedness. Perhaps the iconicity of the lower-level attributions on which such higher-level attributions depend might suffice to render the higher-level attributions iconic. What would have to be the case for this to be so? Suppose entities can be arrayed along various dimensions according to their cohesiveness, rigidity, and boundedness. Spelke-objects would be limit cases, or at least cases sufficiently cohesive, rigid, and bounded. Now consider similarly-arrayed representations of these entities. These representations form a family related in ways that mirror relations among the properties of the entities represented. Suppose further that representations as of cohesiveness, rigidity, and boundedness are iconic. (This might be unobvious—perhaps one would need to consider even lower-level attributions. But bracket this.) The question is whether that would suffice to render attributions of objecthood themselves iconic.8 Not yet. For the attribution of objecthood might involve a representation (a vehicle) distinct from the representation of the lower-level features. Having represented an entity as cohesive, as rigid, and as bounded, perception may then transit to a distinct state that represents it as an object. There's no reason to think the iconicity of the lower-level representations renders the distinct higher-level representation iconic. But what if we suppose further—as an empirical hypothesis—that the state that represents objecthood is the very state that represents the lower-level features? The idea is not to reduce the representation of objecthood to that of the lower-level features. That would be a contentious claim—and we can readily conceive of organisms that represent the lower-level features without the higher-level Block's (2023a, p. point that of attributions do not the is that each state that represents the property is to a state that represents the relevant lower-level It may also have the higher-level content in virtue of the example, downstream effects say, tracking or (As we can an that represents the lower-level features, but not the downstream effects with objecthood would then be A needn't be that perception could transit to a distinct But suppose it The state that represents objecthood would then be it is being that the state represents the relevant lower-level features. This reply on a of (a further one is that all prima facie loners are But, in it has an Representations of on this reply, would be iconic in virtue of the iconicity of other aspects of the representations that represent aspects on which representation of objecthood in part That the reply does not how to objecthood and representations thereof within an The claim is rather that, as it representations as of objecthood are representations that are iconic in other a a by a One consider the attribution of and the of as such iconic just the has other iconic The representational is with both iconic and non-iconic A reply is to to as Block Spelke-objecthood and representations thereof in fact are graded, so that the relevant families are of degrees of objecthood and correspondingly degreed This might seem a to the position of the rather than a distinct reply on a of Block's iconicity But it needn't the reply can be to perceptual loners. That one might ungraded perceptual attributions and the weakened clauses to and also argue that all perceptual are so not loners. could this with that some prima facie cases are perhaps with the highly generic This reply would be to the one could provide a for why all be graded, as opposed to on its to be the case that they To I have some of the that Block's conception of iconicity does not to cases consider This is a Block maintains that perception is constitutively iconic. I have also a of replies and their and in a way that of the view. But it could be that of them What do you … perhaps a with is two distinct an iconic representation that and their and a set of discursive Phillips however, that the of a could be on a by a of out the