Search Results for author:
Found 65 papers, 17 papers with code
Optimal Control Synthesis of Markov Decision Processes for Efficiency with Surveillance Tasks
Our objective is to synthesize a control policy that ensures the surveillance task while maximizes the efficiency.
Prioritize Team Actions: Multi-Agent Temporal Logic Task Planning with Ordering Constraints
Simultaneously, we aim to maintain a pre-determined order in the values of the objective function for each agent, which we refer to as the ordering constraints.
Formal Synthesis of Controllers for Safety-Critical Autonomous Systems: Developments and Challenges
This paper provides a comprehensive review of formal controller synthesis techniques for safety-critical autonomous systems.
A Game-Theoretical Approach for Optimal Supervisory Control of Discrete Event Systems under Energy Constraints
To solve this problem, we propose a game-theoretical approach by converting the cDES as a consumption two-player graph game (cTPG) and reformulate the optimal supervisory control problem in game theory.
Learning Local Control Barrier Functions for Safety Control of Hybrid Systems
Our approach is computationally efficient, minimally invasive to any reference controller, and applicable to large-scale systems.
Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis
One-shot 3D talking portrait generation aims to reconstruct a 3D avatar from an unseen image, and then animate it with a reference video or audio to generate a talking portrait video.
Contribution Functions for Quantitative Bipolar Argumentation Graphs: A Principle-based Analysis
We present a principle-based analysis of contribution functions for quantitative bipolar argumentation graphs that quantify the contribution of one argument to another.
On Approximate Opacity of Stochastic Control Systems
Based on this new system relation, we show that one can verify opacity for stochastic control systems using their abstractions (modeled as finite gMDPs).
Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling
Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting.
Synthesis of Temporally-Robust Policies for Signal Temporal Logic Tasks using Reinforcement Learning
The second objective is to maximize the worst-case spatial robustness value within a bounded time shift.
Signal Temporal Logic Control Synthesis among Uncontrollable Dynamic Agents with Conformal Prediction
We propose a predictive control synthesis framework that guarantees, with high probability, the satisfaction of signal temporal logic (STL) tasks that are defined over the system and uncontrollable stochastic agents.
Sleep When Everything Looks Fine: Self-Triggered Monitoring for Signal Temporal Logic Tasks
Existing works on online monitoring usually assume that the monitor can acquire system information periodically at each time instant.
SoybeanNet: Transformer-Based Convolutional Neural Network for Soybean Pod Counting from Unmanned Aerial Vehicle (UAV) Images
Soybeans are a critical source of food, protein and oil, and thus have received extensive research aimed at enhancing their yield, refining cultivation practices, and advancing soybean breeding techniques.
Safe-by-Construction Autonomous Vehicle Overtaking using Control Barrier Functions and Model Predictive Control
In order to tackle the overtaking task in such challenging scenarios, we introduce a novel integrated framework tailored for vehicle overtaking maneuvers.
NNgTL: Neural Network Guided Optimal Temporal Logic Task Planning for Mobile Robots
In this work, we investigate task planning for mobile robots under linear temporal logic (LTL) specifications.
C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model
Co-speech gesture generation is crucial for automatic digital avatar animation.
Argument Attribution Explanations in Quantitative Bipolar Argumentation Frameworks (Technical Report)
Argumentative explainable AI has been advocated by several in recent years, with an increasing interest on explaining the reasoning outcomes of Argumentation Frameworks (AFs).
Efficient STL Control Synthesis under Asynchronous Temporal Robustness Constraints
Given an ATR bound, we compute a sequence of control inputs so that the specification is satisfied by the system as long as each sub-trajectory is shifted not more than the ATR bound.
Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis
However, the prompting mechanisms of zero-shot TTS still face challenges in the following aspects: 1) previous works of zero-shot TTS are typically trained with single-sentence prompts, which significantly restricts their performance when the data is relatively sufficient during the inference stage.
GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech
Cross-lingual timbre and style generalizable text-to-speech (TTS) aims to synthesize speech with a specific reference timbre or style that is never trained in the target language.
Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions, and Prospects
Conversational recommender systems (CRSs) have become crucial emerging research topics in the field of RSs, thanks to their natural advantages of explicitly acquiring user preferences via interactive conversations and revealing the reasons behind recommendations.
Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis
We are interested in a novel task, namely low-resource text-to-talking avatar.
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies.
Detector Guidance for Multi-Object Text-to-Image Generation
Previous works identify the problem of information mixing in the CLIP text encoder and introduce the T5 text encoder or incorporate strong prior knowledge to assist with the alignment.
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data.
Ranked #7 on
Audio Generation
on AudioCaps
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation
Direct speech-to-speech translation (S2ST) aims to convert speech from one language into another, and has demonstrated significant progress to date.
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation
Recently, neural radiance field (NeRF) has become a popular rendering technique in this field since it could achieve high-fidelity and 3D-consistent talking face generation with a few-minute-long training video.
Data-Driven Safe Controller Synthesis for Deterministic Systems: A Posteriori Method With Validation Tests
We first formulate the safety synthesis problem as a robust convex program (RCP) based on notion of control barrier function.
LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion
As a key component of automated speech recognition (ASR) and the front-end in text-to-speech (TTS), grapheme-to-phoneme (G2P) plays the role of converting letters to their corresponding pronunciations.
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data.
Ranked #11 on
Audio Generation
on AudioCaps
Virtual Try-On with Pose-Garment Keypoints Guided Inpainting
In our method, human pose and garment keypoints are extracted from source images and constructed as graphs to predict the garment keypoints at the target pose.
Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features
Speech-to-speech translation directly translates a speech utterance to another between different languages, and has great potential in tasks such as simultaneous interpretation.
You Don't Know When I Will Arrive: Unpredictable Controller Synthesis for Temporal Logic Tasks
This problem is particularly challenging since future information is involved in the synthesis process.
Markov decision processes with maximum entropy rate for Surveillance Tasks
We consider the problem of synthesizing optimal policies for Markov decision processes (MDP) for both utility objective and security constraint.
Explaining Random Forests using Bipolar Argumentation and Markov Networks (Technical Report)
Random forests are decision tree ensembles that can be used to solve a variety of machine learning problems.
Model Predictive Control for Signal Temporal Logic Specifications with Time Interval Decomposition
However, instead of applying MPC directly for the entire task horizon, we decompose the STL formula into several sub-formulae with disjoint time horizons, and shrinking horizon MPC is applied for each short-horizon sub-formula iteratively.
Abstraction-Based Verification of Approximate Pre-Opacity for Control Systems
In this paper, we first introduce a concept of approximate pre-opacity by capturing the security level of control systems with respect to the measurement precision of the intruder.
Model Predictive Monitoring of Dynamical Systems for Signal Temporal Logic Specifications
To this end, effective approaches for the computation of feasible sets of STL formulae are provided.
Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective
Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information.
A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation
In this paper, we propose a novel Chinese dialect TTS frontend with a translation module, which converts Mandarin text into dialectic expressions to improve the intelligibility and naturalness of synthesized speech.
Towards a Theory of Faithfulness: Faithful Explanations of Differentiable Classifiers over Continuous Data
There is broad agreement in the literature that explanation methods should be faithful to the model that they explain, but faithfulness remains a rather vague term.
A Unified Framework for Verification of Observational Properties for Partially-Observed Discrete-Event Systems
However, in contrast to existing results, where different verification procedures are developed for different properties case-by-case, in this work, we provide a unified framework for verifying all these properties by reducing each of them as an instance of HyperLTL model checking.
A Uniform Framework for Diagnosis of Discrete-Event Systems with Unreliable Sensors using Linear Temporal Logic
In this work, we propose a novel \emph{uniform framework} for diagnosability of DES subject to, not only sensor failures, but also a very general class of unreliable sensors.
Fault Diagnosis of Discrete-Event Systems under Non-Deterministic Observations with Output Fairness
Non-deterministic observation is a general observation model that includes the case of intermittent loss of observations.
To Explore or Not to Explore: Regret-Based LTL Planning in Partially-Known Environments
In contrast to the standard game-based approach that optimizes the worst-case cost, in the paper, we propose to use regret as a new metric for planning in such a partially-known environment.
You Don't Know What I Know: On Notion of High-Order Opacity in Discrete-Event Systems
In this paper, we consider a new type of secret related to the knowledge of the system user.
Sensor Deception Attacks Against Initial-State Privacy in Supervisory Control Systems
Specifically, we consider an active attacker that can tamper with the observations received by the supervisor by, e. g., hacking on the communication channel between the sensors and the supervisor.
Online Monitoring of Dynamic Systems for Signal Temporal Logic Specifications with Model Information
We show that, by explicitly utilizing the model information of the dynamic system, the proposed online monitoring algorithm can falsify or certify of the specification in advance compared with existing algorithms, where no model information is used.
Towards Realistic Visual Dubbing with Heterogeneous Sources
The task of few-shot visual dubbing focuses on synchronizing the lip movements with arbitrary speech input for any talking head video.
Data privacy protection in microscopic image analysis for material data mining
The core contributions are as follows: 1) the federated learning algorithm is introduced into the polycrystalline microstructure image segmentation task to make full use of different user data to carry out machine learning, break the data island and improve the model generalization ability under the condition of ensuring the privacy and security of user data; 2) A data sharing strategy based on style transfer is proposed.
Towards Using Clothes Style Transfer for Scenario-aware Person Video Generation
To tackle this problem, most recent AdaIN-based architectures are proposed to extract clothes and scenario features for generation.
Towards High-fidelity Singing Voice Conversion with Acoustic Reference and Contrastive Predictive Coding
Experiments show that, compared with the baseline models, our proposed model can significantly improve the naturalness of converted singing voices and the similarity with the target singer.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech
In expressive speech synthesis, there are high requirements for emotion interpretation.
Personalized Heterogeneous Federated Learning with Gradient Similarity
In SPFL, the server uses the Softmax Normalized Gradient Similarity (SNGS) to weight the relationship between clients, and sends the personalized global model to each client.
Cloud-Assisted Nonlinear Model Predictive Control for Finite-Duration Tasks
In this paper, we propose a novel cloud-assisted model predictive control (MPC) framework in which we systematically fuse a cloud MPC that uses a high-fidelity nonlinear model but is subject to communication delays with a local MPC that exploits simplified dynamics (due to limited computation) but has timely feedback.
Optimal Synthesis of Opacity-Enforcing Supervisors for Qualitative and Quantitative Specifications
We assume that the system has a "secret" that does not want to be revealed to the intruder.
A Framework for Current-State Opacity under Dynamic Information Release Mechanism
In this paper, we investigate the verification of current-state opacity for discrete-event systems under Orwellian-type observations, i. e., the system is allowed to re-interpret the observation of an event based on its future suffix.
PPG-based singing voice conversion with adversarial representation learning
Singing voice conversion (SVC) aims to convert the voice of one singer to that of other singers while keeping the singing content and melody.
Opacity Enforcing Supervisory Control using Non-deterministic Supervisors
Compared with the standard deterministic control mechanism, such a non-deterministic control mechanism can enhance the plausible deniability of the controlled system as the online control decision is a random realization and cannot be implicitly inferred from the control policy.
End-to-End Learning for Simultaneously Generating Decision Map and Multi-Focus Image Fusion Result
In this work, to handle the requirements of both output image quality and comprehensive simplicity of structure implementation, we propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure.
Xiaomingbot: A Multilingual Robot News Reporter
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation.
Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech
Accent conversion (AC) transforms a non-native speaker's accent into a native accent while maintaining the speaker's voice timbre.
ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders
This paper presents ByteSing, a Chinese singing voice synthesis (SVS) system based on duration allocated Tacotron-like acoustic models and WaveRNN neural vocoders.
A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis
In Mandarin text-to-speech (TTS) system, the front-end text processing module significantly influences the intelligibility and naturalness of synthesized speech.
A hybrid text normalization system using multi-head self-attention for mandarin
In this paper, we propose a hybrid text normalization system using multi-head self-attention.
