An organic brain-inspired platform with neurotransmitter closed-loop control, actuation and reinforcement learning

Abstract

Organic neuromorphic platforms have recently received growing interest for the implementation and integration of artificial and hybrid neuronal networks. Here, achieving closed-loop and learning/training processes as in the human brain is still a major challenge especially exploiting time-dependent biosignalling such as neurotransmitter release. Here, we present an integrated organic platform capable of cooperating with standard silicon technologies, to achieve brain-inspired computing via adaptive synaptic potentiation and depression, in a closed-loop fashion. The microfabricated platform could be interfaced and control a robotic hand which ultimately was able to learn the grasping of differently sized objects, autonomously.

---

New concepts

In this manuscript a brain-inspired closed-loop system has been demonstrated for the accomplishment of motor control and actuation tasks, through a learning process mediated by a neurotransmitter. This system integrates well established silicon technologies with organic materials, more suitable for communication with biological neurons. While the intelligence and the decision-making process is completely delocalized, it lies in the local adaptation of a neuromorphic OECT. Such adaptation is closed-loop controlled using a PID control law, and it mimics the neurotransmitter-mediated synaptic plasticity of biological neural networks (BNNs). From the existing field of research, sensing and motion control have been achieved by exploiting organic neuromorphic architectures, with the goal to recapitulate autonomous local learning typical of the human neural processing (Krauhausen, I. et al. Organic neuromorphic electronics for sensorimotor integration and learning in robotics. Science Advances 7, eabl5068, 2021). In these applications there are two main aspects still missing: the first one is the lack of a neuromorphic control-loop architecture strongly desirable for the adaptive responsiveness to external stimuli of the system; the second one is the employment of biological signalling as responsible of the synaptic plasticity during the learning process, typical of the human brain and useful for the active integration of this technologies in a biological environment. In this new concept dopamine is the signal used for the strengthen of the artificial synapse, integrated in a closed-loop system able of adaptive and reinforcement learning with an object-specific recognition and training.

Introduction

Neuromorphic computing represents an emerging and promising approach in the development of the next generation of hardware. The continuous endeavour of researchers reaches a panoply of different fields, spanning from the design of digital processing architectures, to the development of smart adaptive hardware. In this sense, organic semiconductors, featuring a polymer structure and ionic-electronic mixed conduction properties, have been shown to emulate short- and long-term memory^1–3 when integrated into three terminal devices such as organic electrochemical transistors (OECTs) and arrays.^4–6 Here, these devices were capable of information processing^7,8 classification,^9,10 performing reservoir computing,^11,12 act as fast switching memory elements¹³ and spiking neuronal networks.^14–17 Furtherly, inorganic materials have been largely employed to engineer artificial neurons,¹⁸ with the aim of improving device responsiveness, lowering power consumption and integrating into large arrays. On the other hand, organic materials have recently found major applications as neuromorphic interfaces.¹⁹ In fact, conductive polymers can also provide a biocompatible and stable coupling for biological cells and tissues and therefore support hybrid interfaces between artificial neuromorphic and living neuronal systems.^20,21 Notably, neurohybrid synaptic platforms have been built exploiting electrical communication,²² electrochemical signalling through single and multiple neurotransmitter release^23,24 and direct tissue interfacing and stimulation with implantable probes.²⁵

In addition, neuromorphic architectures, by exploiting neural primitives, may enable real-time interaction with the surrounding environment. Indeed, state-of-the-art neuromorphic controllers may either leverage on neuromorphic models to determine a control law,^26,27 or they can rely on spiking neural networks (SNNs) to implement long-standing control laws, such as proportional, integral and derivative (PID) controllers on silicon neuromorphic chips.^28,29 Crucially, such approaches still fail in recapitulating the autonomous adaptation that characterizes neural processing. As a result, while neuromorphic sensing and motion control in organic neuromorphic architectures were demonstrated,^7,30 a neuromorphic control-loop controlled system is still missing. Indeed, the implementation of a neuromorphic closed-loop architecture is strongly desirable, enabling fully autonomous systems, that could adapt as a response to external stimulation. Here, we present a simple and direct approach in the realization of a neuromorphic closed-loop system, in which silicon and organic materials cooperate to accomplish real world tasks, in real-time. In the presented closed-loop system a silicon microcontroller controls actuation and motion, while the intelligence and the decision-making process is completely delocalized, as it lies in the local adaptation of a neuromorphic OECT. Such adaptation is closed-loop controlled using a PID control law, and it mimics the neurotransmitter-mediated synaptic plasticity of biological neural networks (BNNs). Dopamine (DA) oxidation was exploited to strengthen/weaken the artificial synapse, that controls in real time the opening and closure of a robotic hand in a closed-loop configuration. Finally, autonomous reinforcement learning, based on a reward/punishment protocol is implemented. Here, the organic synaptic device ‘learns’ how to drive the hand to grasp different-sized object. The proposed architecture demonstrated how an organic brain-inspired platform can cooperate with either biological and electronic systems, establishing a neuromorphic closed-loop system.

Results and discussion

The working principle of the neuromorphic device is schematically represented in Fig. 1a. The channel current of the electrochemical neuromorphic organic device (ENODe) was sampled and digitalized, enabling the communication with inorganic hardware, that would eventually drive an actuation.

Fig. 1 Overall neuromorphic closed-loop architecture and open-loop artificial neuron characterization. (a) Schematic of a closed-loop neuromorphic system, coupling a silicon microcontroller to drive external actuation, while delocalizing the decision-making process to the organic synaptic device (left). Comparison between a biological synapse and the organic synaptic device (right). (b) Synaptic potentiation elicited by a train of square voltage pulses applied at the gate terminal in presence of DA. (c) Synaptic potentiation as a function of DA molarity (Numerical values: 3.9 ± 0.3; 4.6 ± 1.9; 6.6 ± 2.2; 10.3 ± 1.3; 19.8 ± 5.5; 21.0 ± 6.7; 18.7 ± 7.1; (N = 3)), along with schematic of the PEDOT:PSS reduction mechanism. (d) Synaptic depression elicited by a train of square voltage pulses applied at the gate terminal in presence of H₂O₂. (e) Synaptic depression as a function of H₂O₂ molarity (Numerical values: 1.2 ± 0.6; 1.8 ± 1.3; 2.6 ± 1.6; 2.4 ± 1.0; (N = 3)), along with a schematic of the PEDOT:PSS oxidation mechanism.

In brief, the neural signalling was recapitulated by applying square voltage pulses at the gate terminal, emulating action potentials (APs) of biological synapses. In a biological synapse, when an action potential reaches the axon terminal of the pre-synaptic neuron, neurotransmitters are released in the synaptic cleft. Such neuroactive molecules, then, bind to specific receptors of the post-synaptic neuron, eliciting an electrical response.³¹

In the organic synaptic device, upon the application of the gate potential, ions would migrate from the electrolyte to the polymeric channel of the transistor, resembling the neurotransmitter release. In addition, such ions would dope/de-dope the polymeric channel of the transistor, modulating its current, and mirroring the post-synaptic potential response.

In addition, continuous stimulation/inhibition of neural connections results in potentiation/depression of specific neural pathways, resulting in the so-called synaptic plasticity.³¹ Here, the ENODe could emulate such synaptic potentiation and depression, by controlling in real time the local concentration of a neurotransmitter. A faradaic reaction was indeed exploited to elicit a charge transfer mechanism, doping/de-doping the channel of the ENODe in a non-volatile way (Fig. 1b, c and d, e, respectively).

Among the variety of voltage-oxidizable neurotransmitters^23,32 that could be employed in the non-volatile conductance modulation of the ENODe channel, dopamine (DA) was chosen, as this molecule is crucial in the closed-loop circuitry of motor-learning in the brain, in which it is used to enforce positive behavioural outcomes, such as the correct execution of a movement.^33,34

First, open-loop non-volatile potentiation and depression (i.e., synaptic plasticity) of the ENODe were demonstrated and characterized. When DA was present in the electrolyte, the application of a gate bias, matching the neurotransmitter's oxidation potential,³² favoured an oxidation reaction and a consequent release of protons and electrons in the electrolyte.³⁵ Cations generated in such reaction would elicit a charge-transfer process to both gate and channel of the organic transistor, reducing the PEDOT:PSS and finally decreasing the device's conductance (Supplementary Discussion S1, ESI†).²⁴ As a result, the redox reaction permanently de-doped the ENODe channel, as the current decreased with the number of pulses (Fig. 1b). The change in the baseline of the current (Fig. 1b) represented the synaptic potentiation of the neuromorphic device, as it mirrored the long-term strengthening of biological synapses that takes place as a response to an increased stimulation.³⁶

Notably, this was a concentration-dependent process (Fig. 1c). A linear dependence of the synaptic potentiation on the concentration of DA employed in the faradaic process was observed in the range 5–30 μM. However, at higher values (50 to 100 μM), the channel conductance modulation would exhibit a saturation behaviour without any significant increase (Fig. S1, ESI†). Here, protons released during the DA oxidation reaction penetrated the bulk of the CP (Fig. 1c), progressively reducing the PEDOT:PSS and de-doping the transistor channel²⁴ whereas the saturation behaviour might occur because of the limited oxidative species present at the gate electrode.²³

In addition, synaptic depression could be recapitulated here by introducing H₂O₂ in the electrolyte solution (Fig. 1d) that oxidizes PEDOT:PSS, reversing the DA-mediated (30 μM) de-doping, restoring the initial conductance level of the ENODe (Supplementary Discussion S1, ESI†).

As shown in Fig. 1e, the conductance variation was not dependent on the concentration of H₂O₂: the synaptic depression slightly increased with increasing H₂O₂ concentration (up to 60 mM), while it remained constant for higher concentrations. Such non-dependency of the synaptic depression on the concentration of the analyte may imply that the H₂O₂-mediated oxidation of PEDOT:PSS was elicited by a reaction occurring at the surface of the material (Fig. 1d; Fig. S2, ESI†).³⁷ Importantly, the difference in the numerical values of synaptic potentiation and depression was correlated to the aforementioned PEDOT:PSS reduction and oxidation mechanisms. In the former, the oxidation of the neurotransmitter could immediately reduce the polymer, dramatically reducing its conductance. In the latter, the H₂O₂ in the electrolyte slowly oxidized the PEDOT:PSS, recovering its conductance.

Subsequently, to achieve real time control of the synaptic potentiation/depression, a y-shaped microfluidic module (see Methods) was coupled to the transistor to regulate the DA and H₂O₂ – gate interface and therefore exploit the flowrate as a control variable. In fact, as previously reported, the flowrate might increase the number of species available for the oxidation, while preventing fouling at the gate electrode.²³ While no significant difference in synaptic potentiation was observed for a 30 μM DA solution under static condition and at low flow rates (0.1 ml min⁻¹), high flow rates (0.5 and 1 ml min⁻¹) induced an almost linear increase of the conductance modulation (Fig. 2a). Furthermore, a linear dependence of synaptic depression was observed when employing the H₂O₂ solution at different flow rates, suggesting that the flowrate increases the amount of solution that actively washed the surface of the polymeric channel (Fig. 2b).

Fig. 2 Artificial neuron closed-loop application and synaptic plasticity control. (a) Synaptic potentiation as a function of the flowrate when 30 μM DA was present in the electrolyte (red trace, numerical values: 10.2 ± 2.4; 14.4 ± 2.5; 15.0 ± 3.8; (N = 3)) and comparison with synaptic potentiation obtained with 30 μM DA under static condition (black circle. Numerical values: 10.3 ± 1.3; (N = 3)). (b) Synaptic depression as a function of the flowrate when 30 mM H₂O₂ was added to the electrolyte (blue trace. Numerical values: 1.9 ± 1.5; 2.5 ± 1.9; 3.4 ± 1.1; (N = 3)) and comparison with synaptic potentiation obtained with 30 mM H₂O₂ under static condition (black circle. Numerical values: 1.8 ± 1.3; (N = 3)). (c)–(e), Numerical simulation of DA and H₂O₂ distribution inside the microfluidic channel at different flowrates ((c), 30 μM DA, 0.1 ml min⁻¹ and 30 mM H₂O₂, 0.9 ml min⁻¹; (d) 30 μM DA, 0.5 ml min⁻¹ and 30 mM H₂O₂, 0.5 ml min⁻¹; (e) 30 μM DA, 0.9 ml min⁻¹ and 30 mM H₂O₂, 0.1 ml min⁻¹), along with numerical simulation of the total DA concentration in the microfluidic channel. (f)–(h), channel current modulation obtained in the same combinations of simulated 30 μM DA and 30 mM H₂O₂ flowrates ((c), (d), and (e), respectively).

Then, to determine the simultaneous effect of DA and H₂O₂ solutions on the synaptic modulation, different flow rates conditions were numerically simulated (Fig. 2c–e) and correlated to the measured channel conductance variation (Fig. 2f–h). Notably, different distributions of DA and H₂O₂ were achieved in the microfluidic channel by changing the ratio between flowrates . When u_DA ≪ u_H2O2, (Fig. 2c)

Thus, the DA-mediated synaptic potentiation (synaptic potentiation ΔG = 9.9%, Fig. 2f) was comparable to the potentiation obtained with same concentration of neurotransmitter under static conditions (Fig. 1c). When , half of the microfluidic channel volume was filled with DA, increasing the synaptic potentiation in comparison to the static case (ΔG = 17.4%, Fig. 2g). Lastly, as u_DA ≫ u_H2O2, DA was present in the whole microfluidic channel, strongly increasing the synaptic potentiation (ΔG = 28.5%, Fig. 2h).

The flow rate could therefore be selected as a control parameter to regulate the long-term modulation of the ENODe and a closed-loop system including an organic synaptic device was implemented (Fig. 3a).

Fig. 3 Closed-loop neuromorphic system and robotic actuation. (a) Block diagram of the closed-loop neuromorphic system. I_DS was measured and compared to a desired setpoint, computing the error of the closed-loop system e and then the control law u, subsequently releasing either DA or H₂O₂, changing their concentration inside the electrolyte of the ENODe, ultimately closing the loop. In addition, I_MEAS was digitalized and sent to the robot, opening or closure the hand. (b) Square voltage pulse applied at the gate terminal during the closed-loop operation, along with the measured current. (c) Closed-loop regulation of the channel current of the neuromorphic device, connected to the robotic hand. Two setpoints (I₁, and I₂, horizontal dotted lines) were chosen as - 235 and - 260 μA, respectively. If the measured current was higher than the setpoint, DA was released to potentiate the synaptic device (red trace), decreasing the amount of current flowing inside the polymeric channel. Conversely, when I_MEAS was lower than I_SET, H₂O₂ was released in the electrolyte (blue trace) achieving synaptic depression. In addition, two tasks were identified (open and closed hand, yellow and green regions, respectively), corresponding to different neurotransmitter's level. As I_MEAS reached the ‘open’ region, the hand is completely open. Conversely, when the current was within the ‘close’ region, the hand was fully closed. (d) Pictures of the open (d-i) and closed hand (d-ii) during the closed-loop measurements.

Here, the completion of a desired task (i.e., hand closure/opening) corresponded to a certain channel current (setpoint), and the competing DA/H₂O₂ flow rates were closed-loop controlled to reach this setpoint. At each loop iteration, the channel current was measured (I_MEAS) and compared to the setpoint (I_SET), to quantify the error (e = I_SET – I_MEAS) of the closed-loop system. This error described how far the actual value of current was from the setpoint. At this stage, a PID controller interpreted the error and computed a control law u, as a linear combination of the error, its integral and derivative over time (Fig. S3, ESI†).

The control law u determined the optimal flowrates of DA and H₂O₂ to minimize the error e. Finally, a square voltage pulse was applied at the gate terminal (V_GS), oxidizing DA (if present in the microfluidic channel), changing the channel conductance and, consequently, the channel current. The current was measured again, determining a new value of I_MEAS, and triggering a new iteration. This process was repeated until the channel current reached the setpoint (e < ε).

Notably, the channel current I_DS was measured after the application of a pulse (Fig. 3b), not including the transient response of the transistor due to ions injected from the electrolyte to the polymeric channel (rising edge of V_GS) and then migrating back to the electrolyte (falling edge of V_GS).³⁸

In addition, the value of I_MEAS was sent to a microcontroller that determined a control signal as an input to servo motors, driving a 3D-printed robotic hand (Fig. S4, ESI†).

The robotic hand could be closed or opened by adjusting I_SET, transducing synaptic potentiation/depression of the ENODe into motor commands: initially, (i.e., before execution of the closed-loop system) the value of I_MEAS sent to the microcontroller was encoded as a complete closure of the hand. If, depending on I_SET, synaptic potentiation occurred, I_MEAS progressively decreased. Such current decrease would be encoded by the microcontroller as the opening command of the robotic hand. Conversely, in case of synaptic depression, the I_MEAS increase would be encoded as a closing command of the hand.

Fig. 3c shows a closed-loop system example where two setpoints were arbitrarily chosen (I₁ and I₂, dashed horizontal lines). Initially, as I_SET was lower than I_MEAS, the system required synaptic potentiation to complete the regulation task. DA was released inside the electrolyte (Fig. 3c), while square voltage pulses were applied at the gate terminal, oxidizing the neurotransmitter, and decreasing the ENODe conductance by reducing PEDOT:PSS. Concurrently, the prosthetic hand gradually opened as I_MEAS decreased, until full opening was achieved (Fig. 3d). Eventually, the measured current I_MEAS decreased, and the regulation task was successfully completed, i.e., I_MEAS ≅ I_SET. Once the value I₁ was reached, I_SET changed to I₂. Here, as the setpoint was higher than the measured current, the system required synaptic depression. The control system responded to such variation by releasing H₂O₂ (Fig. 3c, blue trace), oxidizing the surface of PEDOT:PSS and increasing the channel current, that reached the setpoints (Video S1, ESI†). The workflow of the software for the control is reported in Supplementary Discussion S2 (ESI†).

Finally, by integrating a pressure sensor in the closed-loop architecture (Fig. 4a), reinforcement learning³⁹ was introduced. The goal of this system was to learn to recognize and grasp objects of different sizes, using reward and punishment signals, as shown in Fig. 4a.

Fig. 4 Closed-loop reinforcement learning. (a) Block diagram of the electrical closed-loop for autonomous reinforcement learning. The measured current I_MEAS was digitalized, controlling the grasp of the hand, that was equipped with a pressure sensor. Based on the sensor readout, it was possible to determine either if the hand was able to complete (or not) a task, i.e., grasping an object. In case of failure, DA was released in the electrolyte solution and a punishment electrical signal (square voltage pulse) was applied at the gate terminal, decreasing I_MEAS and further closing the hand. In case of a successful execution of the task, a reward signal was applied (V_GS = 0 V), while removing DA from the electrolyte. (b) Measurements of the electrical closed-loop during a reinforcement learning experiment. (b-i), A tennis ball was employed. If the pressure sensor did not detect the contact with the object (top plot), DA was continuously oxidized in the microfluidic channel, decreasing the channel current (bottom plot) and closing the hand by increasing the angles of the motors (middle plot). The procedure stopped when the grasping is complete. (b-ii) A ping-pong ball was introduced into the system, restarting the learning procedure. (b-iii) The former object (tennis ball) was presented again to the system. Here the grasp was completed instantaneously, as the ENODe had already learned to grasp such object. (c) Pictures of the hand grasping a large object (tennis ball, (c-i)), a small object (ping pong ball, (c-ii)) and the larger object once again (c-iii).

The task presented increased because of the variable object size where diverse degrees of movement were required to adjust and close the hand motors. Initially, the hand was completely open and the current I_MEAS determined the closure of the robotic hand. The extended system could sense the environment through the sensor, describing whether the hand was able to grasp the object or not. In case of a failed grasp, a punishment signal in the form of a square voltage pulse was applied at the gate terminal, oxidizing DA present in the electrolyte of the device. The punishment signal caused I_MEAS to decrease, leading to a further closure of the hand. Conversely, when the hand was able to grasp the object, a reward action was provided, by biasing the gate terminal at zero voltage level, keeping a stable channel current, and preventing a further closure of the hand.

A large (tennis ball) and small (ping pong ball) objects were used in the reinforcement learning experiment. The pressure was continuously read from the sensor to detect contact with the test object, while the angle of the motors was recorded and used as a hand closure parameter. The channel current (measured after each reward/punishment signal), instead, indicated the learning of the neuromorphic ENODe (Fig. 4b).

First, the tennis ball was employed (Fig. 4b-i), and the robotic hand closed (based on the value of I_MEAS) in the attempt of grasping it. The task initially failed (Fig. 4c-i) and consequently a punishment signal was supplied, oxidizing DA, and decreasing I_MEAS. As this process was iterated, the oxidation of the neurotransmitter progressively caused the hand to close (increasing motor angle). When the ENODe learnt how to grasp the tennis ball (i.e., when the pressure sensor detected that the hand correctly touched the object), the reward was provided, and the robot stopped moving (constant motor angle). Then, the ping pong ball was introduced (Fig. 4b-ii). Because the closed-loop system was not trained to grasp such a small object (Fig. 4c-ii), the hand failed in executing the assigned task. Therefore, a punishment signal was supplied, eliciting DA oxidation, and progressively closing the hand (motor angle increases). Punishment signals were continuously provided until the hand completely grasped the smaller ball and completed the learning protocol. Further details are reported in Fig. S5 (ESI†).

Last, the first object (tennis ball) was introduced again. Here, considering that the ENODe had already learned how to grasp it before, the object was immediately grabbed correctly (Fig. 4b-iii and 4c-iii and Video S2, ESI†).

Experimental methods

Device fabrication

ENODe were fabricated on a 25 × 25 mm square glass substrate, with 10 × 10 mm indium tin oxide (ITO) square at each corner (Xinyan Technology Ltd, QGM20210930119). PEDOT:PSS (Hereaus, Clevios PH1000, 81076212) aqueous solution was prepared by adding 5 vol% ethylene glycol (Sigma-Aldrich, 102466-1L-M), 1 vol% (3-glycidyloxypropyl)trimethoxysilane (Sigma-Aldrich, 440167-100ML) and 0.02 vol% dodecylbenzene sulfonic acid (Sigma-Aldrich, 44198-250ML). Glass substrates were treated with oxygen plasma (Tecno-Service) for 2 min at 20 W. Subsequently the PEDOT:PSS solution was spin coated on the substrate at 2000 rpm for 2 min. Thermal annealing at 140 °C on hotplate was performed. PEDOT:PSS gate and channel were patterned through oxygen plasma dry etching technique for 15 min, at 100 W. The physical masks used to define the transistor geometry during the etching procedure were made of polydimethylsiloxane, mixed in ratio 10 : 1 wt/wt with a crosslinker, and cured at 80 °C, 1 h (PDMS, Silgard 184). Finally, through the etching process two symmetrical PEDOT:PSS stripes 7 × 17 mm wide were deposited 2 mm apart. Then the devices were immersed in milliQ water for 1 h to allow for the complete swelling of the PEDOT:PSS prior to further measurements.

Microfluidic system

To allow the electrolyte to flow between the channel and the gate of the ENODe, a microfluidic channel, made of PDMS, was attached on the device by means of uncured PDMS, placed on the edges of the channel, and subsequently cured at 80 °C, 1 h. The microfluidic channel had a rectangular shape (17 × 4 mm) with a y-shaped junction at one end. Three holes were created with the use of a 1.2 mm punch at the ends of the y-shaped channel. Two input holes were connected through microfluidics tubes to two pumps, one for the DA solution and the other one for the H₂O₂ solution. The third output hole was connected to a waste container.

Electrolyte solution preparation

PBS without Ca⁺⁺ and Mg⁺⁺ buffer solution was purchased from Life Technologies, 14190169. DA solution was obtained by dissolving dopamine hydrochloride in powder (Sigma-Aldrich, USA) in PBS. A stock H₂O₂ (30% in water, Sigma-Aldrich, USA) was diluted in PBS to obtain H₂O₂ solutions.

Electrical measurements

All electrical measurements were carried out using a commercially available setup (Arkeo, Cicci Research, Italy) featuring two independent source measure units (SMUs) simultaneously. Pulsed operations were performed by keeping a fixed bias voltage V_DS = −0.2 V, while applying square voltage pulses V_GS at the gate terminal. Each measurement consisted of 6 pulses with amplitude 0.3 V, pulse width 3 s and delay between pulses 9 s.

PID control system

The PID control system was embedded in a LabVIEW routine running on the measurement setup. The proportional coefficient is set to 0.005, the integral one was set to 0.05, while the derivative was set to 0.01. These parameters were found through manual PID calibration. All the details for the software were reported (Supplementary Discussion S2, ESI†).

DA static measurements and data analysis

100 ml DA solution was inserted in the microfluidic coupled to the channel of the ENODe. Each measurement consisted of 6 voltage pulses applied at the gate terminal. After each measurement three washes were performed with 100 ml of fresh PBS solution. The channel conductance was calculated by dividing the channel current by V_DS. Then, conductance variation (difference between values before and after the application of V_GS pulses) was represented as percentage.

Three consecutive measurements were performed, resulting in the sum of 18 applied voltage pulses, and conductance variations were computed and averaged for each device. At least N = 3 devices were measured with this procedure to obtain the results shown in the manuscript.

H₂O₂ static measurements and data analysis

H₂O₂ static measurements were carried out as described in Methods (DA static measurements and data analysis). The only difference in the procedure is that one measurement with DA (6 voltage pulses) was performed before the application of H₂O₂.

Data analysis

Data analysis was carried out through custom made MATLAB scripts.

Numerical simulations

Numerical simulations were carried out through COMSOL Multiphysics 6.0, by coupling “laminar flow” and “transport of diluted species” built in modules.

Robotic hand printing and electrical actuation

The hand was 3D printed using an open-source cable-driven design (InMoov, https://inmoov.fr/). Hand closure was achieved by driving servomotors (one per fingers). Cables were mounted on the motors so that the 0-degree position corresponds to completely open hand, while 90-degree position corresponds to a fully closed hand. Commands were sent to the servomotors through a commercial microcontroller board (Arduino Uno board). A custom-made Arduino script was used to read data through serial interface from the electrical measurement setup and drive the servomotors. When performing reinforcement learning, a feedback signal was sent by serial interface to punish or reward ENODe (Fig. S4 and S5, ESI†).

Conclusions

Neuromorphic systems featuring innovative materials are emerging in the never-ending quest for the next generation of hardware. In this sense, closed-loop processing is essential, as it represent the foundation of countless real-time and real-world tasks. In light of this, closed-loop neuromorphic system are advisable, as they could potentially lead to fully autonomous and adaptive systems. Here, we presented a simple and direct approach to endow long-standing closed-loop architectures with neuromorphic capabilities. Organic synaptic devices emerged as unique and singular option in such approach, as they can communicate with silicon-based technologies, while providing adaptive and smart features, ultimately leading to the complete delocalization of the intelligence of the system. As a result, while a microcontroller dealt with simple read/write operation through serial interfaces, the organic chips adapted to complete the required task. In perspective, by reducing the size of the organic neuromorphic devices and optimizing the geometrical ratio (W/L),^6,40 the switching speed of the devices could be reduced, allowing to match biologically plausible operating speed. In addition, as the adaptation closely mirrors biological potentiation/depression paradigms, and it is based on biologically relevant signals, such approach may represent the first step towards smart prosthetics devices and adaptive clinical approaches.

Author contributions

U. B and D. R. contributed to the experimental design, data collection, analysis, and manuscript writing. C. A, A. M., S. M., O. B., C. L. supported experimental design, F. S. contributed to project and experimental design, data discussion, manuscript revision and funding acquire.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

F. S. and D. R. acknowledge funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (BRAIN-ACT, Grant agreement No. 949478). F. S., C. L., S. M. acknowledge funding from the Exploratory Research Space ERS Funds (‘NeuroWin’, OPSF705).

Notes and references

1. P. Gkoupidenis, N. Schaefer, B. Garlan and G. G. Malliaras, Neuromorphic Functions in PEDOT:PSS Organic Electrochemical Transistors, Adv. Mater., 2015, 27, 7176–7180.
2. Y. van de Burgt, et al., A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing, Nat. Mater., 2017, 16, 414–418.
3. P. Gkoupidenis, Y. Zhang and H. Kleemann, et al., Organic mixed conductors for bioinspired electronics, Nat. Rev. Mater., 2024, 9, 134–149.
4. S. T. Keene, P. Gkoupidenis and Y. van de Burgt. in 18 - Neuromorphic computing systems based on flexible organic electronics, Organic Flexible Electronics, ed. Cosseddu, P. & Caironi, M., Woodhead Publishing, 531–574, 2021 DOI:10.1016/B978-0-12-818890-3.00018-7.
5. Y. van de Burgt, A. Melianas, S. T. Keene, G. Malliaras and A. Salleo, Organic electronics for neuromorphic computing, Nat. Electron., 2018, 1, 386–397.
6. J. T. Friedlein, R. R. McLeod and J. Rivnay, Device physics of organic electrochemical transistors, Org. Electron., 2018, 63, 398–414.
7. I. Krauhausen, et al., Organic neuromorphic electronics for sensorimotor integration and learning in robotics, Sci. Adv., 2021, 7, eabl5068.
8. D. A. Koutsouras, et al., An Iontronic Multiplexer Based on Spatiotemporal Dynamics of Multiterminal Organic Electrochemical Transistors, Adv. Funct. Mater., 2021, 31, 2011013.
9. Y. Zhang, et al., Adaptive Biosensing and Neuromorphic Classification Based on an Ambipolar Organic Mixed Ionic–Electronic Conductor, Adv. Mater., 2022, 34, 2200393.
10. J. Y. Gerasimov, et al., A Biologically Interfaced Evolvable Organic Pattern Classifier, Adv. Sci., 2023, 10, 2207023.
11. M. Cucchi, S. Abreu, G. Ciccone, D. Brunner and H. Kleemann, Hands-on reservoir computing: a tutorial for practical implementation, Neuromorph. Comput. Eng., 2022, 2, 032002.
12. M. Cucchi, et al., Reservoir computing with biocompatible organic electrochemical networks for brain-inspired biosignal classification, Sci. Adv., 2021, 7, eabh0693.
13. Y. Tuchman, T. J. Quill, G. LeCroy and A. Salleo, A Stacked Hybrid Organic/Inorganic Electrochemical Random-Access Memory for Scalable Implementation, Adv. Electron. Mater., 2022, 8, 2100426.
14. P. C. Harikesh, et al., Ion-tunable antiambipolarity in mixed ion–electron conducting polymers enables biorealistic organic electrochemical neurons, Nat. Mater., 2023, 22, 242–248.
15. T. Sarkar, et al., An organic artificial spiking neuron for in situ neuromorphic sensing and biointerfacing, Nat. Electron., 2022, 5, 774–783.
16. G.-T. Go, Y. Lee, D.-G. Seo and T.-W. Lee, Organic Neuroelectronics: From Neural Interfaces to Neuroprosthetics, Adv. Mater., 2022, 34, 2201864.
17. P. C. Harikesh, et al., Organic electrochemical neurons and synapses with ion mediated spiking, Nat. Commun., 2022, 13, 901.
18. G. Indiveri, et al., Neuromorphic Silicon Neuron Circuits, Front. Neurosci., 2011, 5 DOI:10.3389/fnins.2011.00073.
19. D. V. Christensen, et al., roadmap on neuromorphic computing and engineering, Neuromorph. Comput. Eng., 2022, 2, 022501.
20. M. Cucchi, D. Parker, E. Stavrinidou, P. Gkoupidenis and H. Kleemann, In Liquido Computation with Electrochemical Transistors and Mixed Conductors for Intelligent Bioelectronics, Adv. Mater., 2023, 35, 2209516.
21. U. Bruno, et al., From neuromorphic to neurohybrid: transition from the emulation to the integration of neuronal networks, Neuromorph. Comput. Eng., 2023, 3(2), 023002.
22. I. Tzouvadaki, P. Gkoupidenis, S. Vassanelli, S. Wang and T. Prodromakis, Interfacing Biology and Electronics with Memristive Materials, Adv. Mater., 2023, 2210035.
23. G. M. Matrone, et al., Electrical and Optical Modulation of a PEDOT:PSS-Based Electrochemical Transistor for Multiple Neurotransmitter-Mediated Artificial Synapses, Adv. Mater. Technol., 2023, 8, 2201911.
24. S. T. Keene, et al., A biohybrid synapse with neurotransmitter-mediated plasticity, Nat. Mater., 2020, 19, 969–973.
25. T. Wang, et al., A chemically mediated artificial neuron, Nat. Electron., 2022, 5, 586–595.
26. R. Halaly and E. Ezra Tsur, Autonomous driving controllers with neuromorphic spiking neural networks, Front. Neurorobot., 2023, 17, 1234962.
27. C. M. Niu, et al., Neuromorphic Model of Reflex for Realtime Human-Like Compliant Control of Prosthetic Hand, Ann. Biomed. Eng., 2021, 49, 673–688.
28. F. Galluppi, et al., Event-based neural computing on an autonomous mobile platform. in 2014 IEEE International Conference on Robotics and Automation (ICRA) 2862–2867 (2014). 10.1109/ICRA.2014.6907270.
29. T. DeWolf, et al., Neuromorphic control of a simulated 7-DOF arm using Loihi, Neuromorph. Comput. Eng., 2023, 3, 014007.
30. W. Wang, et al., Neuromorphic sensorimotor loop embodied by monolithically integrated, low-voltage, soft e-skin, Science, 2023, 380, 735–742.
31. E. R. Kandel, J. H. Schwartz, T. M. Jessell, S. A. Siegelbaum and A. J. Hudspeth, Principles of Neural Science, McGraw-hill, New York, 2000, vol. 4.
32. I. Gualandi, et al., Selective detection of dopamine with an all PEDOT:PSS Organic Electrochemical Transistor, Sci. Rep., 2016, 6, 35419.
33. A. R. Wang, A. Groome, L. Taniguchi, N. Eshel and B. S. Bentzley, The role of dopamine in reward-related behavior: shining new light on an old debate, J. Neurophysiol., 2020, 124, 309–311.
34. R. G. Lewis, E. Florio, D. Punzo and E. Borrelli, The Brain's Reward System in Health and Disease, Adv. Exp. Med. Biol., 2021, 1344, 57–69.
35. R. P. Bacil, L. Chen, S. H. P. Serrano and R. G. Compton, Dopamine oxidation at gold electrodes: mechanism and kinetics near neutral pH, Phys. Chem. Chem. Phys., 2020, 22, 607–614.
36. R. G. M. Morris and D. O. Hebb, The Organization of Behavior, Brain Research Bulletin, Wiley, New York, 1949. vol. 50, p. 437, 1999.
37. D. Felder, et al., Coupled Ionic–Electronic Charge Transport in Organic Neuromorphic Devices, Adv. Theory Simul., 2022, 5, 2100492.
38. P. R. Paudel, M. Skowrons, D. Dahal, R. K. Radha Krishnan and B. Lüssem, The Transient Response of Organic Electrochemical Transistors, Adv. Theory Simul., 2022, 5, 2100563.
39. L. P. Kaelbling, M. L. Littman and A. W. Moore, Reinforcement Learning: A Survey, J. Art. Intell. Res., 1996, 4, 237–285.
40. D. A. Bernards and G. G. Malliaras, Steady-State and Transient Behavior of Organic Electrochemical Transistors, Adv. Funct. Mater., 2007, 17, 3538–3544.

---

Footnotes

† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d3mh02202a

‡ These authors contributed equally.

§ Current address: Institute of Polymers, Composites and Biomaterials (IPCB) − CNR, Viale J. F. Kennedy 54, Mostra D'Oltremare Pad 20, Naples, 80125 Italy.

¶ Current address: Department of Materials Science and Milano-Bicocca Solar Energy Research Centre (MIB-SOLAR), Via Cozzi 55, I-20125, Milano, Italy.

|| Current address: The International Clinical Research Center of St. Anne's University Hospital in Brno (FNUSA-ICRC), Pekařská 53, 656 91 Brno-střed, Cechia.

---